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ABSTRACT 



This manual, which is intended for vocational education and 
training (VET) providers, does the following: explains fundamental market 
research concepts; highlights situations where market research can help VET 
providers with their planning and policy development; and illustrates the 
uses of market research in VET institutions. The following topics are 
discussed: purposes of market research; situations where market research is 
useful (defining problems, determining information requirements, and deciding 
between in-house and commissioned market research) ; market research briefs 
(format and content) ; data for market research (existing data, new data, and 
sampling) ; sources of existing data (internal sources, external sources, and 
14 public agencies providing statistical data about VET in Australia) ; 
collection of qualitative data (situations where qualitative data are useful, 
methods, and collector skills for qualitative research) ; collection of 
quantitative data (survey methods, questionnaire design, and questionnaire 
samples) ; and management of the research process (field work, data 
processing, data analysis, and report writing) . Concluding the manual are 
five case studies focusing on conducting research to determine the following: 
best way(s) to use promotions budgets; reasons for declining enrollments; 
feasibility of raising fees for short courses; student satisfaction with 
courses or programs; and an institution's image in the community. The book 
contains 13 references. (MN) 



******************************************************************************** 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

******************************************************************************** 




ic: ■■■■-' 

rK-'-f 









:-,Xi; 



) \: 



(rlfeiSfe ,„ 

III 

||f| W'-W-\ ' XX; 



U.S. DEPARTMENT OF EDUCATION 
Offioe of Educational Research and Improvement 
CATlONAL RESOURCES INFORMATION 
CENTER (ERIC) 

Yhis document has been reproduced as 
f received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



lii* 



sMb* m 

y'- Mh: \ M;-45 



V : $» 




MARKET 



RESEARCH 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 



MANUAL 



mi: 



=cjf ftoirfe r c x | t, . : 'i 







For Providers of 

Vocational Education and Training 



OTFE 






Published by the Office of Training and Further Education 
525 Collins Street 
Melbourne Vic 3000 
Australia 

Tel: (03) 9628 2333 

Fax: (03) 9628 3966 

© State of Victoria 1996 

Requests and inquiries concerning reproduction and copyright should 
be addressed to the General Manager, Strategic Planning and Research, 
Office of Training and Further Education, 525 Collins Street, 

Melbourne Vic 3000, Australia. 

The views expressed in this book do not necessarily represent the views 
of the Office of Training and Further Education. 

National Library of Australia cataloguing-in-publication data: 

Tamblyn, Annette. 

Market Research Manual for Providers of Vocational Education 
and Training 

Bibliography. 

ISBN 0 7306 9312 0 

1. Vocational education - Research - Victoria. 

2. Occupational training - Research - Victoria. 

3. Marketing research - Methodology. 

I. Shelton, Diane. 

II. Victoria. Office of Training and Further Education. 

III. Title. 

374.01309945 



Acknowledgements 

This manual was written by Annette Tamblyn and Diane Shelton of Chant 
Link and Associates as part of a project for the Office of Training and 
Further Education, Victoria. 

The publisher thanks Associate Professor Gerald Burke, 

Centre for the Economics of Education and Training, Monash University, 
for his assistance in preparing the directory on pages 33-40. 

The Office of Training and Further Education thanks all those who took 
part in the research to ensure that the manual would be useful to providers 
of vocational education and training. 

o 

ERLC 



Contents 



Preface 



v 



Introduction 1 

Purposes of market research 2 

The market research process 3 

When should market research be used? 5 

Problem definition 6 

Information requirements 6 

Should market research be used in this case? 6 

In-house or commissioned research? 6 

Market research brief 9 

Format 1 0 

Sample brief 13 

Data for market research 1 7 

Existing data 1 8 

New data 1 8 

Sampling 20 

Sources of existing data 25 

Internal sources 26 

External sources of statistical data 26 

Directory 27 

Collecting qualitative data 35 

When is qualitative data useful? 36 

Methods of collecting qualitative data 37 

Collector skills for qualitative research 39 

Collecting quantitative data 43 

Survey methods 44 

Questionnaire design 46 

Questionnaire samples 59 



ERJC pav 



*3 



iii 



Managing the research process 67 

Field work 68 

Processing data 69 

Analysing data 71 

Reporting 72 

Case studies 75 

1 How can we best use the promotions budget? 76 

2 A case of declining demand 82 

3 Should we raise the fees for our short courses? 87 

4 How satisfied are our students? 91 

5 What is the image of our institute in the community? 96 

References 1 03 



O 

ERLC 



Preface 




TT his manual was commissioned by the Office of Training and Further 
Education to give providers of vocational education and training an overview 
of the role and importance of market research and a guide to why, when and 
how to use market research. 

Senior managers with little experience of market research will appreciate 
this overview of ways in which market research can be used to advantage. 
Staff with responsibility for commissioning or conducting market research 
can use the manual as a text or a reference. It is not designed to be a step- 
by-step guide to conducting market research. Market research needs to be 
tailored to the specific needs of the organisation that requires marketplace 
information. 

The main aims of the manual are to: 

• familiarise non-specialists with fundamental market research concepts; 

• indicate suitable contexts for market research processes for providers of 
vocational education and training; 

• highlight situations in which market research can help providers of 
vocational education and training with their planning and policy 
development; 

• identify a range of situations for employing market research within 
vocational education and training organisations; 

• provide illustrative case studies of market research undertaken by 
providers of vocational education and training. 

The format and content of the manual have been developed as the result 
of a number of depth interviews and group discussions with senior 
management and marketing personnel of TAFE institutes and private 
provider institutions in regional and metropolitan Victoria. These 
discussions, themselves a form of qualitative research, provided insights into 
the current status of market research amongst providers of vocational 
education and training, and an understanding of the range and level of 
market research topics that were important to them. On completion of the 
draft version of the manual, sample workshops were held to pilot the manual 
with users. Participants were drawn from the earlier discussion groups. 



If 



V 



Introduction 




8 



1 




R C H ma nvJ* u 



r he market for vocational education and training has changed markedly 
over the last five years. An increased number of providers, changes in the 
types and expectations of students and other clients, and changes in state 
and federal government policy, have all contributed to a competitive market 
for vocational education and training. 

Universities, private providers and TAFE institutes now all compete for 
students. Students have a greater number of choices and are increasingly 
aware of their power to choose. 

Providers of vocational education and training have to attract and satisfy 
a number of client groups: not only students undertaking training but their 
sponsors, employers, government agencies, parents, enterprises that 
commission training and the community in general. Community 
expectations are particularly important in regional locations. 

The rapid pace of change in vocational training has left many people 
unsure of the relative merits of different suppliers of vocational education 
and training. Hence, there is considerable confusion in the marketplace 
about the role of providers of national education and training within the 
overall system. 

Government policies now favour open competition between providers to 
determine who will deliver vocational education. All providers, including 
TAFE institutes are now competing with other suppliers to ‘sell’ their 
training and educational services to a range of clients including large 
government funding bodies, industry groups, individual companies and 
corporations and individual students. 

Market research can help institutes to manage and optimise opportunities 
in this complex and heterogeneous market. Market research can increase the 
capacity for informed decisions in an environment where intuition and 
experience may no longer suffice. 

Please note that the use of the term ‘ institutes' in this manual refers to 
both public and private providers of vocational education and training. 
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EPOSES OF PESEAPOOO 

Market research is a formalised, systematic approach to gathering and 
analysing information about your market in order to make sound 
management decisions. It can be used by providers of vocational education 
and training to provide valuable information for: 

• management decisions, whether as direct input or to raise questions that 
need to be addressed before making a decision; 
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• better understanding of the education needs of students and other client 
groups, of their attitudes towards your institute, of how they choose one 
institute over another and of how their attitudes and values affect their 
choice; 

• knowing what business enterprises require in terms of both course content 
and course delivery mechanisms, so as to be able to extend relationships 
already built up with local business to a wider business market; 

• better understanding of preferred delivery modes, for example student and 
employer preference for study at home, at an institute or at a workplace; 

• testing client satisfaction, for example with the range of programs and 
services that you are offering - an important issue given the importance 
of return business in this market and the importance of word-of-mouth 
recommendations; 

• evaluation of programs and services, for example using results of 
outcome-focused research with former students and their employers, to 
justify continued support by government or private enterprise; 

• communication strategies for publicity and promotion, for example to 
pinpoint whether the most effective channel of communication to your 
target markets is the local paper, the city paper, the radio or television; 

• development of the institute’s image, for example to understand how 
student, employer, community and government perceptions of your 
institute and its image should affect your decisions about name changes, 
signage, colours or logos; 

• knowing your competitors and your position in the market relative to 
them, which is particularly useful for distinguishing the real threats from 
the supposed ones; 

• pricing of services to ensure that you are competitively priced and that 
you are receiving the optimum price for your services; 

• new market opportunities. 

TME [VO^KCCET RESE/S\[R!€M PROCESS 

The process for conducting market research involves a number of key stages: 
investigating, planning, data collection, analysis, reporting and action or 
implementation. Each of these stages is discussed in detail in the following 
chapters. The market research process is outlined on the following page. 
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THE MARKET RESEARCH PROCESS 



1 Investigating Define the problem or issue to be investigated 

What kind of information is required? 

What key questions need to be answered? 

I 

Should market research be used? 



If so, should it be conducted in-house or by an 
external agency? 



2 Planning 



Market research brief 



3 Data collection ^ — 

Obtain data from 
existing sources 



Develop or select data- 
gathering instruments 



Administer instruments, 
make observations 



4 Analysis Analyse data, document information 



5 Reporting Disseminate information to audiences 



6 Action Audience makes judgements, decisions 
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When 
should 
market 
research 
be used ? 5 




5 




IVIarket research is not always appropriate or necessary. Your collective 
knowledge may be sufficient for you to make a good decision. In many 
organisations, intuition and experience have long been the most common 
bases for decision making, and have often proved to be sufficient. In 
considering the use of market research, adopt the following process. 

F>B(0)B[LE[M] ©EFOMO¥0®M 

State the problem explicitly. Market research is not a ‘quick fix’ or a panacea 
for all problems. It is important to be explicit about the main aim of the 
research, for example: 

• Why are enrolments declining in key programs? 

• Does the institute need to invest in open access learning technology? 

• How can we best use the publicity budget to promote the institute 
effectively? 
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0MF©F1[M]A¥D©[M REQUIREMENTS 

State the information requirements. What kinds of information would help? 

What questions need to be answered? How would the needed information 
be best sought? Alternatives to market research include intuition or personal 
knowledge, experience, and consultation with experts. 

SCHOOL© 0¥i]$\F3G€E¥ KESE^RCC=3 BE USED DM ¥MQS CASE*? 

Compare the value of the research with the cost of the research (see next 
chapter for a costing guide). If you are making decisions about where a new 
campus is to be situated, where the outlay for the campus is in the order of 
millions of dollars, then the cost of the project warrants spending some 
money on research to ensure that the decision is the right one. However, 
when deciding on course or program design, the research might be useful 
but might not warrant the cost. 

0M-IH]©ySE ©Fi ©©[MHM3DSSDOME© FIESEAFSCM'? 

Market research can be conducted by institute staff or by an external agency 
or by both. The decision will be influenced by the research aims, the 
institute’s resources, funds available, the time line for the research and the 
availability of appropriate external resources. 

Providers of vocational education and training usually conduct their 
research in-house. However, apart from a few TAFE institutes, most 
providers have had little experience of formal market research. Most 
marketing decisions have been made on the basis of analysis of existing data, 
and on sound intuition and experience. 

Now, however, as institutes operate beyond their traditional markets, face 
increased competition and pursue higher quality courses, they need 

13 J BEST COPY AVAILABLE 




sophisticated and extended market research studies. They need to consider 
whether it is efficient and effective to conduct their market research in-house 
or to commission an external agency to undertake the work. The following 
issues deserve consideration. 

• Cost: There is a strong perception that the costs of using an external 
agency for market research are beyond the resources of most providers 
(see next chapter for a costing guide). Externally commissioned research 
can certainly be costly, but needs to be compared with the actual costs, 
including the many hidden costs, of in-house research. 

• Other resources: Have you the necessary resources within the institute 
for the project? Although many institutes will have staff skilled in various 
aspects of research, they may not be available to work on the project 
within the set time frame. 

• Nature of the research: Some forms of market research need to be carried 
out by external agents because the nature of the project requires a degree 
of objectivity that internal providers cannot manage, particularly if the 
issue is controversial, it may be sensible to get an external agent to 
conduct projects that are politically sensitive, for example research on 
renaming an institute, or reviews of programs or of organisational units 
within the institute. Other types of research, such as forecasting local 
demand, may best be conducted in-house because you probably know far 
more about the nature of the research than an external agent could be 
expected to know. 

• Degree of control required: Some researchers are reluctant to use external 
agents because they prefer to keep all the research and information under 
their control. It is true that control is delegated when a project is 
outsourced. It is therefore important to form the right kind of relationship 
with the selected external agency. 

• Availability of appropriate external agents: Providers of vocational 
education and training operate in a complex and complicated marketplace 
that has recently become very competitive. Some providers doubt the 
capacity of most external market research agencies to understand the 
context in which they are wishing to conduct the research. They also 
assume that it would be too time consuming to ‘instruct’ the external 
agencies about the market for vocational education and training. 

• Experience: Lack of experience with outsourcing market research prevents 
some people using it. They often assume that it is going to be too difficult 
to manage, too costly, unnecessary and not as effective as conducting the 
research in-house. 
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IN-HOUSE OR COMMISSIONED RESEARCH? 

1 Expense/value Is it worth the extra cost of using an 



2 Objectivity/credibility 

3 Political expediency 

4 Broader perspective 

5 Confidentiality 

6 Professionalism/ 
expertise 

7 Retain control 

8 Economies of time 
and technology 

9 Understanding of 
the market 
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external agency? Will you be getting value 
for money? 

Is there a need for objectivity in the study? 

Are there any political reasons why an 
external agent may be preferable? 

Are you looking to broaden your perspective 
on the issue? 

Is there a need for confidentiality? 

Do you have the necessary expertise and 
personnel to manage or conduct the project? 

Is there a need/preference to retain control of 
the research project? 

Are there likely to be economies of time and 
technology in using an external agency? 

Is there a need for a high level of 
understanding of the vocational education 
and training market? Do the available 
agencies have this understanding? 
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T his planning stage is necessary whether you are to conduct the research 
yourself or whether you are to commission someone else to do it. It is 
difficult to overestimate the importance of this stage of the process. If you 
state the overall requirements and objectives of the research clearly at the 
outset you are likely to get the information you want from the research. 

[Backgjimwinidfl 

The extent of background information required will depend on who is 
conducting the research and how familiar they are with the context of the 
research. However, even if you are conducting research in-house, it is unwise 
to assume shared knowledge. From the outset it is important to create shared 
understanding of the background to the research among those 
commissioning the research, those conducting it and those who will act on 
its outcomes. 

The background information should define the problem that faces the 
decision makers by addressing the following questions: 

• What are the factors that have led to the need to make a decision? 

• What information is needed for making the decision? 

• How will the information be used? By whom? 

Keseaircfh) ©tbjjecttn^es 

Research objectives need to be clear, unambiguous and concisely stated. 
They shape the design, analysis and reporting of the research. Research 
objectives should state the key questions the research needs to answer and 
the kinds of information to be sought, such as: 

• geographic scope - specific metropolitan, regional or rural areas; 

• respondent scope 

- students (current students, students who have a high level of 
experience of your institute, new students, prospective students, 
gender, age) 

- parents 

- employers 

- government officials (local, state, federal?) 

- general community. 
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The quality of the research is influenced by the amount of time allowed for 
it. Ensure that there is enough time to do the research properly. A rushed 
piece of research can be counterproductive. Allow sufficient time to get the 
right person to do the research. It can be worth waiting for the best researcher 
available. 

Schedule the various elements of the research to occur at appropriate 
times. Arrange data collection to occur at a time when your respondents are 
going to be accessible. For issues with only periodic interest for respondents, 
collect the data when respondents are concerned with those issues, for 
example it would be best to ask students or other clients about institute 
choices at a time when they are thinking about the issues. Otherwise you 
will get only theoretical responses. Don’t plan to collect data from students 
during school or university holidays as they will be away. Avoid business- 
focused research at the end of the financial year when the people you want 
to talk to will not be available. 

Cosftotrcgj arad lb inCOg eft 

The value of the information to be collected should not exceed the cost of 
collecting it. Specify your budget limits and expectations. If your planning 
process indicates that the research is beyond your budget, it may be better 
to abandon the research proposal than to spend less money on research that 
will not match your needs. 

However, the most expensive research is not always the best one for the 
purpose. Cost should be balanced by a sound understanding of the value of 
the information to be collected and the importance of getting the 
management decision right. 

INDICATIVE COSTINGS FOR MARKET RESEARCH > 

3 

The following estimates, compiled in 1995-96, are subject to change. x 

m 
— i 

Consultants 

x 

m 

The cost of a research consultant can vary from between $600-$1 200+ per 8-hour w 

day, according to market forces, expertise, prestige, and specialist capacity for a ™ 

niche market. x 

o 

x 

Case studies co 

x 

The cost of a case study varies according to the number of issues to be investigated ^ 

and the number of consultations within one case study site. Case studies can cost 
between $3000-$5000 per study or per site. 



University researchers 



Annual salaries, indicating levels of expertise, are listed below. To determine rough 
costs of employing these staff for shorter times, identify the length of time it is anticipated 
employment may be required for the level of staff expertise desired. Then calculate 
the costs on a pro rata basis. 

Principal Research Fellow 
Senior Research Fellow Level D 
Senior Research Fellow Level C 
Research Fellow Level B 
Research Fellow/Assistant Level A 

Casual rates per hour 

Research Assistant Level A 
Research Assistant 2 
Research Assistant 1 

Mail survey 

Costs are calculated according to the necessary sample size, the response rate required 
and postage and labour costs for the following tasks: 

• Advance notice letter 

• Questionnaire accompanied by a stamped addressed envelope for its return 

• Any necessary additional follow-up contacts or financial incentives designed to 
increase the response rate 

• Data entry, coding, analysis, report writing. 

Telephone surveys 

Costs are calculated according to the necessary sample size, the response rate required 
and the labour costs of interviewers, based on: 

• Estimated time of each interview 

• Estimated time required to set up the interview arrangements and any follow-up 

• Writing up the interview transcript 

• Travel costs 

• Analysis, report writing. 

Depth interview and focus groups 

The cost for a consultant to conduct a depth interview is about $500-$600 per 
interview. A consultant will charge about $2000-$3000 per focus group. 

Further indicative costings can be obtained by contacting organisations that conduct 
market research. It is also helpful to discuss these issues with others in your institute 
or organisation who have undertaken similar activities. 



$80,176 

$62,242-$68,571 
$51 ,692-$59,605 
$42,198-$50,1 1 1 
$29,539-$40,087 

$21.88 

$18.56 

$16.37 








• criteria on which the success of the research will be judged; 

• the nature of the report and other products required and how they are to 
be used, possibly including information about number of copies of the 
final report needed, timing of other related research, the need for an 
interim presentation of results; 

• who will be using the products and the format required. 

Projectt mm a du a® ©mm© du tt 

The research brief should specify who is responsible for the overall 
management of the project from beginning to end and who is to manage all 
communication between the researchers and the end users of the research. 
Communication linkage on schedules, focus and interim findings is very 
important during the life of a research project. Misunderstandings need 
cause only a minor interruption if they are uncovered and rectified as early 
as possible in the research process, rather than emerging as major problems 
only after the report is written. 

MARKET RESEARCH BRIEF 



• Background 

• Research objectives and questions 

• Research schedule 



• Costing and budget 

• Performance expectations and evaluation 

• Project management responsibility 



S^IMHPLE EKDEF 
IP roll ©eft 

What drives our clients’ satisfaction? 




[SaickgjtroMinid] 

ABC institute is a large multi-campus college located in the fast growing 
north-eastern corridor of Melbourne. Our services to students include 
diploma courses, short courses, industry-based courses and certificate 
courses. 
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To meet our objectives in relation to growth and planning requirements 
we need to develop a measure of student/client satisfaction. We would like 
to develop a client satisfaction monitor. 

However, before we can monitor client satisfaction we have to determine 
what drives client satisfaction and also where ABC is currently positioned 
in terms of providing that satisfaction. 

ABC already conducts client satisfaction surveys on new clients/students, 
but mainly to evaluate the performance of our promotions and enrolment 
staff. Questions in those surveys relate to the effectiveness of our publicity 
in attracting the student, the efficiency of our enrolment staff in meeting 
student needs and how satisfied students are with the institute’s 
performance so far. 

We have also conducted one-off ad hoc surveys at various times that have 
included components of client satisfaction. 

Related research 

Research was conducted for us in 1993 by XYZ market research on drivers 
of client satisfaction. It provided ABC with a hierarchy of the needs of its 
students/clients. These are listed below in order of importance: 

• Range of courses 

• Flexibility in enrolment 

• Timing of lectures/classes 

• Good instructors 

• Industry contact 

• Good student resources - library, canteen, child minding 

• Friendly environment. 

Note 

Since the XYZ study in 1993, ABC’s concept of ‘client’ has been extended 
to include anyone to whom we are marketing our services: not only the 
students who undertake our courses but also parents of many students, 
employers who encourage or subsidise or fund training at our institute, and 
governments. In the research to be undertaken now, ‘client’ needs to be 
defined in a way that acknowledges this extended concept. 

Ri© search ©tbjjecfto^©© 

1 To identify what areas of service, product delivery and distribution, 
operation and administration are important to ABC clients’ satisfaction 
with the institute; 

To identify our clients’ perception of ABC’s performance in each of these 
areas; 
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To provide standard measures of client satisfaction that can be used over 
time to provide comparative data. 

2 To determine who our clients see as being our main competitors. 

Notes 

1 The views of all ABC clients are needed. Because of the diversity of our 
business, we wish to know about client satisfaction according to the 
courses or programs in which they are enrolled, for example: 

• Diploma courses 

• Certificate courses 

• Leisure courses 

• Short courses. 

There may be further ways of segmenting our client group so as to obtain 
valuable information. This issue should be discussed before the research 
commences. 

2 As we are a major post-secondary non-university institute, many people 
assume that we compete directly with other post-secondary non- 
university institutes. However, we also compete with private providers, 
universities and corporate programs in all the areas in which we offer 
courses and programs. We are interested in developing our understanding 
of the nature of this competition. 

3 This research activity - to identify the drivers of client satisfaction - is 
the first stage of a three-stage project. The second stage will be directed 
at determining the relative importance to clients of each of these drivers 
(a quantitative study). In the third stage a tracking system will be 
established for internally monitoring the performance of ABC on these 
service drivers, and also the performance of our competitors. 

ScDuecOtuiOe 

Results are expected within 6 weeks. 



$25,000 

[Perfforinniairoc© expjecttatfcDOinis atrocfl 

Performance on this project will be evaluated on the basis of the following: 

• The research objectives are met. 

• A clear, succinct report has been written providing information and 
solutions. The report will be ready for printing and will be presented on 
disk in the prescribed format. 



• The research process has enhanced relationships between ABC institute 
and its client. 

• Timelines are met. 

p>o-<a>jj©<Dlt maGTiag©[nro<3[rDft 

The project will be managed by a steering group chaired by the deputy 
director of ABC institute. Day-to-day project management will be the 
responsibility of the selected consultant. 

Project management relationships and project milestones will be set down 
and agreed in the project contract to be signed by both parties. 
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entral to the design of the market research project is the kind of data to 
be collected. In many cases, sources of existing data can be tapped to provide 



necessary to collect new data in order to meet the research objectives, 
researchers collect either qualitative data, quantitative data, or a combination 
of the two. 

EXOSTOMO DATA 

Market research can often rely on information already collected. Your own 
institute routinely collects and collates information from students and 
inquirers, for instance their age, address, course preferences and future 
employment goals. 

Published information in the form of government statistics, reports and 
other printed material can often go some (or all) of the way to answering 
the information needs of market research. Sources of this kind of data 
include Australian Bureau of Statistics, Department of Planning and local 
councils (demographic statistics), Department of Education, Employment, 
Training and Youth Affairs, Commonwealth Employment Service, industry 
bodies (statistics on employment trends) and Office of Training and Further 
Education (statistics on student enrolments). The next chapter provides 
further information on sources of existing data. 

MEW [DATA 

Data collection has to be undertaken in cases where existing data is not 
available, is not relevant to your needs, or is out of date. Data can be 
qualitative or quantitative. 

Qualitative research methods are used to explore issues and provide 
important insights into what people are thinking or feeling. They help 
you to get ‘close’ to the market and to obtain an objective view of issues 
such as: 

• How would students like their courses to be presented? 

• What do business clients think of our institute as a place to send their 
employees? 

• How do people make decisions about which institutes to choose? 

• What motivates a student to choose a particular course? 

Typical tools of qualitative research are depth interviews (with one or two 
people) and focus group discussions with relatively small samples of people. 
The results are often reported in the form of direct quotes that give the 
essence of what people think about the issues raised. A later chapter deals 
with collecting qualitative data. 



sufficient information for meeting the research objectives. Where it is 




Quantitative research methods are used to measure responses to issues being 
investigated, for example: 

• Fifty per cent of respondents were satisfied with the service and 40 per 
cent were dissatisfied (10 per cent ‘don’t know’). 

• Thirty per cent of Gippsland farmers are interested in a particular short 
course, program or service. 

With the appropriate use of sampling, quantitative tools can reliably 
estimate the extent to which particular views are held by respondents 
representing target populations. The most typically used tool of quantitative 
market research is the questionnaire-based survey, conducted by an 
interviewer in a face-to-face interview or by telephone, or by self-completion 
(in a lecture theatre or by mail). 

C^omlbrnattiioinis 

The two approaches are not mutually exclusive. They are often used 
together, for instance depth interviews can be a useful way of refining the 
questions for a questionnaire survey. 



TYPES OF DATA USED IN MARKET RESEARCH 



Data needed for market research 



Use existing collections Collect new data 



Surveys 



Qualitative Quantitative 

methods methods 



Depth interviews 
Focus groups 
Mini groups 
Case studies 
Observation 



Mail surveys 
Telephone surveys 
Face-to-face surveys 
Other 
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It is usually wise to precede quantitative research with some qualitative 
research. You need to have explored the issues - to know what they are, to 
know people’s perceptions about these issues, to know the language they 
use - before measuring responses. If your resources are limited, scale down 
the quantitative research rather than leave out the qualitative, unless you 
are looking for very specific and simple information. 



QUALITATIVE OR QUANTITATIVE METHODOLOGIES? 

• Do you need to measure responses to particular questions - or to understand 
processes? 

• Do you need to measure the strength of certain opinions or perceptions - 
or find out whether they matter? 

• Can the issues be successfully translated to a questionnaire? 

• How will the respondent react to a questionnaire? 

• Will the respondent complete the questionnaire personally? 

• Is a large sample needed? (How variable is the population and how accurate 
do you need to be?) 



S^[MD[P[L0M<G 

Some data is collected on the basis of a census, for example you might survey 
every student undertaking a particular course. However, to survey everyone 
in a population is generally too costly in terms of time, money and 
personnel. A good sample can provide valid and reliable information. 

Sampling is a complex technical matter (for a full discussion, see Kalton). 
However, key issues affecting good sampling practice are addressed below. 

SsunrotpO© sose 

The following issues influence the size of the sample to be taken, (for a 
mathematical discussion of sample size determination see Weiers and also 
Dillon et al). 



< Cost 

* 2 . 

Cost is a critical issue. However, if you can’t afford the cost of the necessary 
sample size, don’t do the research. Consider cheaper data collection methods 




to maintain sample size, for example a telephone survey rather 
to-face survey. Alternatively, maintain the sample size but collect fewer 
items of data, for example there may be no need to have separate categories 
for male and female. 

Degree of accuracy required 

The greater the need for accuracy, the larger the sample size needed. As a 
very general rule of thumb, you need a random sample of 400 of a targeted 
population to get results that are within 5 per cent of accuracy. 

The greater the variability of response within the targeted population, the 
bigger the sample required. The following table illustrates the point. 




ACCURACY OF RESPONSE (%) 

by sample size and level of agreement in target population 



Sample size 


Level of agreement in targeted population 

90% 80% 70% 60% 50% 


600 


2.4 


3.2 


3.7 


3.9 


4.0 


250 


3.7 


5.0 


5.7 


6.1 


6.2 


100 


5.9 


7.8 


9.0 


9.6 


9.8 



The table shows that when most people give the same response (when 
90 per cent agree), you can obtain accuracy of +/- 5.9 per cent with a sample 
as small as 100. But when they give highly variable responses (only 
50 per cent agreement), the error increases significantly (to +/- 9.8 
per cent) for samples of 100. 

Type of research approach 

The type of research you are conducting will influence the size of sample 
that you will require. Quantitative approaches usually require fairly large 
samples to compare the incidence of opinions, attitudes or behaviours. For 
such purposes, statistical formulae which take into account the sample size 
and the variation in the population, can be used to calculate the level of 
accuracy of the results. These formulae can also be used to calculate the 
sample size needed to achieve reliability in your results. Readily available 
statistical packages such as Systat use such formulae. 

Qualitative approaches seek to explore issues such as decision processes 
in order to produce results which are indicative rather than conclusive. 
Qualitative samples are usually small, so need to be carefully selected. 
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FACTORS DETERMINING SAMPLE SIZE 



Factors indicating a large sample 

1 Decisions to be based on the survey data have very serious or 
costly consequences. 

2 There is a need or a demand for a very high level of accuracy in 
the data. 

3 There is likely to be a high level of variability amongst the respondents 
to be surveyed. 

4 The sample is to be divided into relatively small sub-samples during 
analysis and interpretation. 

5 Project costs and timing vary only slightly with increases in the size 
of the sample. 

6 Time and resources are readily available to cover the costs of data 
collection. 
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Factors indicating a small sample 

7 Few if any major decisions or commitments are to be based on the 
survey data. 

8 Rough estimates of results are sufficient for present purposes. 

9 The population to be sampled is very homogeneous, with little variability 
between respondents. 

1 0 Costs increase dramatically with sample size. 

1 1 Budget constraints or time requirements limit the volume of data that 
can be collected. 
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There are many ways of generating a sample. This section provides a broad 
overview of some of the more common ways of doing so. 



Non-probability sampling 

In this most common form of sampling used in market research, samples 
are selected by some non-random process such as the judgement of the 
researcher. Non-probability sampling includes: 

• Quota sampling where the aim often is to obtain a sample that is similar 
to the targeted population in some characteristics such as age or sex. 
Often Australian Bureau of Statistics data are used to decide what quotas 
to apply. Sometimes the researcher makes a judgement based on the 
information required (half the interviews with people who have done a 
short course in the last 12 months and half with people who have never 
done short courses). 

• Convenience sampling where the sample is selected for its convenience 
and low cost for the researcher. Asking students to volunteer to take part 
in research or intercepting students as they leave the cafeteria would be 
examples of convenience sampling. 

• Snowball sampling where initial respondents are selected randomly, but 
where additional respondents are obtained from referrals from the initial 
respondents. This form of sampling is useful if respondents are hard to 
find. 

With non-probability sampling, it is usually not possible to calculate 
sampling error (which tells you how accurate your results are). So you have 
to consider how to avoid introducing bias into the sample - or how to take 
bias into account in interpreting the results. For example, if you do 
telephone interviews to homes during the day, you would not be able to say 
your results were representative of the population because, in the main, you 
would be interviewing retired, unemployed and home duties people. If you 
asked students to volunteer to take part in a survey, you may find that only 
those who held strong views or who were very interested in the subject took 
part. 

Probability sampling 

With probability sampling, each element of the targeted population has a 
known chance of being selected for the sample. The sampling is done by 
mathematical decision rules. The advantage of probability sampling is that 
it is bias-free and permits the measurement of sampling error. It is often 
difficult to conduct probability sampling in-house, as you need to have lists 
of all sample elements, systems to help you randomly generate the sample, 
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and special systems to ensure the sample remains random. However, many 
external agencies have the capability to do probability sampling. Two of the 
most common forms of probability sampling are: 

• Simple random sampling in which each sample element has a known and 
equal probability of selection. Some research agencies use a random 
telephone number generator in telephone studies. 

• Stratified random sampling in which the overall population is divided 
into strata (such as people who have undertaken post-secondary studies 
and those who have not), and a random sample is generated from within 
those strata. 

Sample design needs to be complex and precise. For a full coverage of 
this topic, consult some of the handbooks listed in the References. 
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Information needs can often be met by using the many rich sources of 
existing data available to the researcher. Tapping sources of existing data is 
usually faster and less costly than engaging in collecting new data. Sources 
of excellent data can be found both within your institute and externally. 
However, even promising sources of existing data may not give you the 
breakdown of information you want. You may want data for a specific 
industry or a particular geographic area, but find you can access only 
aggregated information. Sometimes the data may not prove to be recent 
enough for your needs because of the time lapse between collection and 
becoming available in published form. 

SOdJJPSCES 

Your own institute routinely and systematically collects information from 
students, staff and even people who simply inquire about courses. Examples 
include demand data (take-up of courses or inquiry levels for different types 
of courses) and demographic profiles (of students or inquirers). Much of this 
type of information is collected through staff use of a uniform instrument 
for collecting the data (usually a simple data sheet which staff can quickly 
fill out), and a means of systematically collating and analysing the data. 
However, internal data is collected for a range of purposes, some of it not 
sufficiently rigorously to be relied upon for formal research purposes. 

SOURCES OF SYATTOSTTOCAIL DATA 
Below is a directory of excellent public sources of statistical data for 
vocational education and training. Researchers may find the geo- 
demographic mapping service of the Australian Bureau of Statistics of 
particular value. Geodemographic data can provide information about a 
population in a particular geographic area. Geodemographic ‘maps’, 
available from the Australian Bureau of Statistics, draw on the considerable 
data collected via its censuses and surveys to promptly tailor information 
for your specific purposes. 

Contact the National Account Manager, Market Research and Management 
Consulting, Tel (03) 9615 7781, Fax (03) 9615 7798. 

CommeircoaO dlatta [bases 

Access to commercial electronic data bases can be gained via a personal 
computer, modem and appropriate software (for a full discussion, see 
Johnson). Thousands of such data bases are now available through a range 
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of different systems. Access is usually by subscription. There are five 
categories of commercial data bases: 

• Bibliographic: indexing of publications 

• Financial: detailed financial information about companies 

• Statistical: demographic, econometric data for forecasting and doing 
projections 

• Directory/encyclopedia: factual information about people, companies and 
organisations 

• Full text: a full document can be printed out (for a discussion, see 
Green et al). 

Data from all the above sources can sometimes be difficult to interpret 
or can be open to misinterpretation. It is wise always to check the source of 
the data and how and when it was collected. If possible, you should cross 
reference the information with other sources or, at the very least, examine 
how it fits with what you know, and whether it seems to make sense. 

BORECYORY 

Selected data sources for vocational education and training 

National and Victorian bodies responsible for providing statistical data 
relevant to vocational education and training are listed below, along with 
their key publications. Additional unpublished data are available from many 
of these sources. 

Australian Bureau of Statistics 

5th floor, Commercial Union Tower, 485 LaTrobe Street, Melbourne Vic 3000 



Tel (03) 9615 7000 

PO Box 1 0, Belconnen ACT 261 6 c 

Email address stat.info@abs.telememo.au D 

Tel (06) 252 6007 ™ 

0 ) 



PUBLICATIONS 

Guides and overviews (catalogue numbers) 

A Directory of Education and Training Statistics 1995 (1 136.0) 



An Index of Information about Women (1 1 1 7.0) -/ 

A Guide to the Australian National Accounts (5235.0) z 

Q 

Directory of Labour Market and Social Survey Data 1 994 (1 1 35.0) 

o 

Education and Training in Australia (4224.0) > 

Australian Social Trends 1995 (4102.0) > 





27 



Catalogue of Publications (1 101 .0) 

Directory of Industrial Relations Statistics (1 134.0) 

Information Paper: ABS Business Register - Recent Developments (8130.0) 
Small Business in Australia (1 321 .0) 

Year Book Australia (1 301 .0) 

Census 

Directory of Census Statistics (2910.0) 

1991 Census Data Quality: Education 
1996 Census Nature and Content (2008.0) 



Employer surveys 

Employer Training Expenditure Australia, July-September 1993 (6353.0) 
Employer Training Practices Australia, February 1994 (6356.0) 



Longitudinal survey 

Employment and Unemployment Patterns 



Regular labour force and supplementary surveys 

The Labour Force (6204.0) 

Transition from Education to Work, Australia (6227.0.40.001) 

Participation in Education, Australia (6272.0.40.001) 

Career Experience, Australia (6254.0.40.001) 

Labour Force Status and Other Characteristics of Migrants, Australia (6250.0.40.001) 
Characteristics of Small Business Owners, February 1995 (8127.0) 

Employment Benefits (6334.0) 

Job Search Experience of Unemployed Persons, Australia (6222.0.40.001) 
Underemployed Workers (6265.0.40.001) 

Labour Mobility, Australia (6209.0) 



Special supplementary surveys 

1 990 Survey of Income and Housing Costs and Amenities, Persons with Earned 
Income, Australia (6546.0) 

Training and Education Experience, Australia 1993 (6278.0) 

Incidence of In-House Training in the Private Sector (6278.0.40.001 and 002) 
Career Paths of Persons with Trade Qualifications, Australia (6243.0) 

Adult Literacy Survey (Survey in 1 996) 

Disability, Ageing and Carers, Australia, Summary of Findings (4430.0) 

Household Expenditure Survey (6530.0) 

The Effects of Government Benefits and Taxes on Household Income (6537.0) 
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Population Survey Monitor, Adult Education and Training, May 1995 (4103.0) 
National Aboriginal and Torres Strait Islander Survey 1 994 (41 90.0) 
Commercial Training Providers, Australia 1994 (6352.0) 

Household Expenditure Survey (6535.0 and 6537.0) 



Other collections 

Schools Australia (4221 .0) 

Australian Demographic Statistics (3101 .0) 

Estimated Resident Population by Sex and Age: States and Territories of Australia 

(3201 .0) 

Australian National Accounts, National Income and Expenditure (5204.0) 
Expenditure on Education, Australia (551 0.0) 

Industry Profiles (8223.0) 

Labour Force Projections, Australia (6260.0) 

Projections of the Populations of Australian States and Territories, 1993 to 2031 

(3222.0) 

Industry Training Profiles , prepared for Australian National Training Authority 

Australian Council for Educational Research 

Private Bag 55, Camberwell Vic 3124 
Tel (03) 9277 5555 
Fax (03) 9277 5500 
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PUBLICATIONS 
Longitudinal surveys 

Youth in Transition 
Australian Youth Survey 

Australian Committee on Vocational Education and Training 
Statistics 

Level 3, 95 Northbourne Avenue, Braddon ACT 2612 
Tel (06) 246 9141 



PUBLICATIONS 

Quarterly data from state and territory training authorities 

Australian Training Statistics, States and Territories 
Australian Training Statistics, National Summary 
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Annual aggregate state and territory revenue and expenditure data 

Collection of National Financial Data on Vocational Education and Training 



Employer survey 

National Employer Satisfaction (for release 1996) 



Survey of destinations of 1994 TAFE graduates collected by the Australian 
Bureau of Statistics 

Graduate Outcomes , Technical and Further Education, Australia 1995 

Australian National Training Authority 

AMP Place, 10 Eagle Street, Brisbane Qld 4000 
Tel (07) 3246 2300 
Fax (07) 3246 2490 

PUBLICATIONS 

Client and financial 

Annual National Report 

Department of Employment, Education, Training and 
Youth Affairs 

PO Box 9880, Canberra ACT 2601 
Tel (06) 240 8111 



PUBLICATIONS 

Guide and overview 

Annual Report 

<£ Portfolio Budget Measures 



Monthly, various data sources 

Australian Regional Labour Market Assessments 
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Secondary analysis of demographic, labour force, educational and 
qualification data collected by the Australian Bureau of Statistics 

Australia’s Workforce in the Year 2005 



Annual summary statistics from several sources 

Education at a Glance 



Quarterly 

Australian Jobs Review 
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Small Area Labour Markets 



Estimates of labour force variables for labour force regions 

Australian Regional Labour Markets 



Annual data on overseas students 

Overseas Student Statistics 



Six-monthly summary of short and long term prospects 

Job Futures 



Trends in employment, vacancies and earnings 

Labour Market Trends and Prospects 



Collections from higher education institutions 

Selected Higher Education Statistics 



Six-monthly bulletin, based on a variety of sources 

Skills Demand Bulletin 



Monthly count of newspaper advertisements 

Skilled Vacancy Survey 



Results of survey of labour demand by the Australian Bureau of Statistics 
and summaries of trends in vacancies recorded by the Skilled Vacancy 
Survey and Commonwealth Employment Service 

Short Term Prospects Bulletin 

Department of Industrial Relations 

GPO Box 9879, Canberra City ACT 2601 
Tel (06) 243 7333 
Fax (06) 243 7933 
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PUBLICATIONS 

Survey of managers and union delegates 1989-90 and 1995-96 

Australian Workplace Industrial Relations Survey, Technical Report and Data Release 
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Directorate of School Education 

Rialto South Tower, 525 Collins Street, Melbourne Vic 3000 
Tel (03) 9628 2222 

Freecall for anywhere in Victoria 1800 809 834 

PUBLICATIONS 

Produced annually with several relevant tables 

Summary Statistics of Victorian Secondary Schools 

National Centre for Vocational Education Research 

252 Kensington Road, Leabrook SA 5068 
Tel (08) 332 7822 
Fax (08) 331 9211 



PUBLICATIONS 

Annual data from all providers of vocational education and training that 
report through state and territory authorities 

Selected VET Statistics 



Office of Training and Further Education 

GPO Box 266D, Melbourne Vic 3001 
Tel (03) 9628 2333 
Fax (03) 9628 3966 



PUBLICATIONS 

Enrolment data and student contact hours cross-tabulated by streams and 
fields of study, colleges, and other factors 

TAFE College Statistics 



Instructions to providers within the Victorian state training system for 
collecting 1995 student statistical data 

Student Statistical Collection, 1995 



Information, particularly by industry, to guide training providers for funding 
identified by the Office of Technical and Further Education 

« Industry Training Priorities: Victorian State Training System 1995-96 

LU 

cn 

h Listing of details of registered private providers in Victoria, including the 

^ scope of their registration 

ct State Register of Private Providers of Vocational Education and Training 

^ 
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Australian Vice-Chancellors Committee 



GPO Box 1142, Canberra ACT 2601 
Tel (06) 285 8200 



PUBLICATIONS 
Annual survey 

Survey of Applicants for Undergraduate Higher Education Courses 

Commonwealth Grants Commission 

5-7 Torrens Street, Braddon ACT 2601 
Tel (06) 275 3000 
Fax (06) 275 8021 



PUBLICATIONS 
Estimates based on state data 

Report on General Revenue Grants Relativities 

IBIS Group 

Level 3, 1 Collins Street, Melbourne Vic 3000 
Tel (03) 9650 2166 
Fax (03) 9650 4033 



PUBLICATIONS 

Analysis of range of data from Australian Bureau of Statistics 

Industry Outlook Service 

Syntec Economic Services Pty Ltd 

123 Lonsdale Street, Melbourne Vic 3000 
Tel (03) 9662 1911 

PUBLICATIONS 

Analyses of data from Australian Bureau of Statistics 

Guide to Growth 

Ministerial Committee on Education, Employment, Training 
and Youth Affairs 

141 Rathdowne Street, Carlton Vic 3053 
Tel (03) 9639 0588 



PUBLICATIONS 

Collections from schools and school authorities 

National Report on Schooling in Australia and Statistical Annex 
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Collections from government school authorities 

Summary Finance Statistics from the Government Section of the National Schools 
Statistics Collection 

Graduate Careers Council of Australia 

PO Box 28, Parkville Vic 3052 
Tel (03) 9344 4666 
Fax (03) 9347 7298 



PUBLICATIONS 
Annual survey 

Graduate Destinations Survey 

State Training Board of Victoria 

GPO Box 266D, Melbourne Vic 3001 
Tel (03) 9628 2333 
Fax (03) 9628 3966 



PUBLICATIONS 

Data concerning activities, achievement and policy directions for the 
Victorian State Training System. 

Annual Report 



Your local council is another source of labour market research and local 
economic forecasting. It is worth a telephone call to find out what 
information they have. Other research done by institutes and providers of 
vocational education and training is unpublished. It can be useful to inquire 
whether anyone else has done the kind of research that you are considering. 
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qualitative 

data 
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Qualitative data is collected in order to understand people’s processes, 
attitudes and behaviour. It describes rather than counts. Qualitative data is 
usually obtained through conducting depth interviews, group discussions, 
case studies or systematic observation. Qualitative data is characterised by 
lengthy interaction with small samples of respondents. The data is presented 
in respondents’ own words. 

Collecting qualitative data can be more expensive than collecting 
quantitative data, because well trained interviewers may need to spend 
considerable time with respondents. Qualitative data also needs to be 
analysed by trained analysts. It is not possible to enter qualitative data into 
a software program and push a button for the results. 

Qualitative data is often collected and analysed to reveal what sorts of 
questions should be directed to questionnaire respondents in a later study 
that collects quantitative data. The qualitative data collection (and later 
analysis) can ensure that the survey is based on questions that really matter 
to the researcher. 
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WHEM IS QUALBTATDVE ©AM (USEFUL*? 

The major purposes and applications of qualitative research are described 

below. 

• Exploration: Qualitative research is an excellent tool for developing 
understanding of complex processes, perceptions, values and attitudes. 
For example, it would be the best tool for developing a model of how 
students and other clients make decisions about which institute to attend. 
Analysis of such qualitative data would enable you to understand not only 
the primary motivations for their decisions but also their decision 
processes, the major influences on their decisions, when decisions are 
made, and what role the institute itself can play in facilitating the 
decisions. 

• Idea generation: Qualitative data can be effective in generating new ideas. 
If your institute is aware of the need to structure and deliver its courses 
differently but unsure of options, qualitative data could help with 
developing options and gauging preferences for them. 

• Understanding motivation: In attempting to understand ‘underlying’ 
motives for choice of institute, qualitative data can help you to move 
beyond the pragmatic reasons for the choice of institute to understanding 
the deeper motivations for choice. Depth interviewing would be the most 
effective way to probe a respondent’s underlying motivations. 

• Description: You might want to understand or be able to describe how 
students use the institute, the extent to which they make use of facilities 





and services, or how they are helped or hindered in their use of the 
institute’s services. Qualitative data can present a suitable range of 
responses, potential behaviour or attitudes that can then be quantified by 
means of a large-scale survey. 

lETMOOS OF COD-LECTQMO ©UJAILinrAYOVE 
[©©pjftlh] fitrotioir^Gows 

Depth interviews (also called semi-structured interviews) explore issues 
with one person at a time. Depth interviews are good for exploring issues 
that are important to the respondent and developing an understanding of 
individual decision-making processes. The practice and theory of depth 
interviewing is well discussed by Jones and also by Gordon and Langmaid 
(chapter 6). 

A depth interview typically takes about an hour to conduct. The 
interviewer records responses in detailed verbatim notes. A tape recorder 
is sometimes used to help in this process. 

A depth interview must be a conversation, not a set of questions. Depth 
interviews have minimal obvious structure, the idea being to learn what the 
respondent thinks and knows about an issue, not to ask them specific 
questions. Interviewers have a set of major issues that are to be discussed 
in the interview - an interview guide. However, if an interview becomes just 
a series of responses to questions that you have already formulated, you will 
learn nothing new about what is important to the respondent. The real value 
of a depth interview is to find out about the issues that you could not predict 
without conducting the research. 

Depth interviewing requires an expert interviewer who is able to assist 
the thinking processes of the respondent, keep them relatively on track and 
pursue relevant lines of inquiry that emerge during the interviewer 
‘conversation’. The interviewer must also be able to write up the interview 
results in an ‘objective’, perceptive and methodical way. 

Focgjs SJiroGJp)© 

Focus groups comprise 6 to 10 people who address a topic for between one 
and three hours under the ‘soft’ guidance of a moderator. Although groups 
can be as small as 4 for specific purposes, group dynamics work best with 
6 to 8 people. 

Focus groups are used primarily for interaction between participants on 
a topic of interest to the research. However, they may also be used to study 
group processes. Groups can be used to pilot questionnaires. 
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The role of the moderator or facilitator is an important one. Rather than 
leading the group, the moderator informally encourages discussion amongst 
the group, suggests topics and keeps a balance between following the 
discussion guide (prepared by the researcher) and letting the conversation 
flow freely. The group’s discussion is recorded on audio tape or video tape. 
For a discussion of the dynamics and process of focus groups see Goldman 
& McDonald, Hedges, and Gordon & Langmaid (chapters 3—4). 

Organisational issues 

When using focus groups, the following organisational issues need attention 
and decision. 

• Affinity versus non-affinity groups: An affinity group is a group of people 
with an affinity of some kind. They may share a similar outlook on the 
issues under discussion, or perhaps a similar background. In non-affinity 
groups the individuals tend to have little in common with one another. 
Is it better for your research to have groups comprised of affinity groups 
or non-affinity groups? 

Affinity groups will be easier to manage initially than non-affinity groups, 
but there is a danger that the participants will agree with each other and 
become focused on the social aspects of the discussion rather than on the 
ideas being discussed. Non-affinity groups can be difficult to manage. In 
extreme cases the chances of a useful dialogue are limited. 

The purpose of the focus group will provide guidance as to which kind 
of group should be used. For example, a group comprising both students 
and their teachers may be unsuitable for an open and honest discussion 
about problems in course delivery but a good forum for discussing general 
problems facing the institute. 

• Location: Where will you hold the group discussion? For an affinity 
group, it is often best to go to a location where respondents are 
comfortable, such as the staff room for teachers. Holding a group 
discussion between students and teachers in the staff room, however, 
would create an immediate disadvantage for the students. Holding it in 
the student union would create a problem for teachers. Try to hold non- 
affinity group discussions on neutral territory. 

• Previous group experience: Some people often volunteer for group 
discussions because they enjoy the process of participating in group 
discussions. However, experienced ‘groupies’ are often not typical of the 
population that you are wanting to learn more about. Ensure that your 
results are not being biased by the presence of too many participants 
motivated more by wanting to share their opinions than by a genuine 



interest and involvement in the topic. To reduce the number of 
use a standard recruiting procedure that screens out potential participants 
who have attended a group discussion in the last six months. 

• Incentive: What incentive will you use to encourage people to come along 
to the group discussion? In consumer research participants are typically 
offered money or gifts for participation. In vocational education and 
training contexts, participation might be encouraged simply through being 
very direct about the aims of the project and promising to inform 
respondents of the results of the project. 

• Taping the session: As it is not possible to facilitate a group discussion 
and to take notes at the same time, you will need to record the session 
on audio tape or video tape, for later transcribing and analysis. 




COLLECTOR SCCDfLD-S FOR QUJAILDYATDWE RESEARCH 

Both depth interviews and group discussion require the use of highly skilled 
and trained interviewers. Gordon & Langmaid (chapters 5 and 7) provides 
a good discussion of interviewing techniques. A good interviewer needs to 
be interested in people. While it is possible to train people to be 
interviewers, if they do not have a basic interest in other people it will be 
difficult for them to become very good at it. To be effective, an interviewer 
must take into account the following: 

• Accurate empathy: The interviewer needs to be able to accurately perceive 
and communicate, with sensitivity, the feelings and experiences of 
another person, and to understand their meaning and significance. The 
interviewer needs to be able to understand the respondent’s perspective 
on the topic even if it is very different from their own. 

• Sincerity: The interviewer’s responses must be sincere, not phoney. The 
interviewer should not patronise the respondent and should be genuinely 
interested in what the respondent has to say. 

• Non-judgemental: The interviewer must accept the respondent’s 
behaviour without imposing conditions. It is not the interviewer’s role 
to make judgements about the values and beliefs of respondents. 



©ojestiGorntrog] ttecIhiiniocQQJies 

The art of asking questions is central to obtaining good qualitative data from 
interviewing. The open-ended question is the most commonly used form of 
question in this sort of interviewing. It is designed to encourage the 
respondent to talk at any length they choose. 
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For example, the open-ended question, ‘Why did you choose to come to 
this institute?’ does not impose any limits on the response. The respondent 
could talk about any aspect of their decision to come to the institute ranging 
from practical concerns, ‘it was on a bus route’, to more subjective personal 
reasons, ‘I like the “feel” of the institute’. There is no specific form of answer 
required. 



Use off sBOetroce 

The prime reason for the interview is to find out about the respondent’s 
thoughts, feelings, attitudes and ideas. This will not happen if the 
interviewer is doing too much of the talking. Silence is a powerful technique 
in interviewing. Used well it can help generate effective discussion. 

However it is probably the most difficult skill to learn to use when 
collecting qualitative data through interviewing. Few of us are comfortable 
with silence and we tend to ‘fill up’ silences. However, an interviewer who 
fills the silences won’t learn about respondents’ views. Ideally interviewers 
should speak for less than 10 per cent of the interview time. The following 
tactics are helpful for managing silence. 

• Ask a question and then be quiet (ask the question and count to 60 under 
your breath). 

• Tape your interviews to check if you are speaking for less than 10 per cent 
of the time. 

• Become aware of both your own emotional state during silence and the 
state of your respondents. 

• Monitor your own levels of anxiety. Interviewer anxiety is reflected in 
interviewers talking too much and answering their own questions. 

• To facilitate discussion without speaking, use minimal encouragers such 
as nods, smiles and other forms of body language. 
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Set a good foundation for an interview or a discussion by making the 
participants feel at ease and feel that the activity is valuable and worth their 
participation. The following procedures will help. 

• Explain the purpose of the research. People are generally not disposed 
to trust you and be frank and open with you if the purpose of the exercise 
is shrouded in mystery. There would be very few situations in which an 
institute would carry out research requiring that participants be unaware 
of the purpose of the project. 

• Begin with direct questions and move on to open-ended questions. 
For example, begin with ‘How far did you have to travel to get here?’ 




before getting to ‘How do you feel about the courses that the institute 
offers?’ The direct questions make it easy for people to respond first of 
all while the open-ended questions elicit the information that you really 
want to get. 

• Listen well. The importance of listening may seem very self-evident. 
However, it is surprising how often moderators and interviewers can 
easily miss an important opportunity to open up the discussion because 
they are thinking about the next question rather than listening to the 
current response. Listening properly may require giving up some levels 
of control of the interviewing process and being prepared to follow the 
discussion as it naturally unfolds. 

• For group discussions, arrive early so that you personally greet all 
participants on their arrival. 

• Start group sessions by getting everyone present to talk informally about 
some aspect of themselves. This ensures that all participants have spoken 
at least once before moving onto the issues for discussion. Invite questions 
or comments from the group at any time. 
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ollection of quantitative (number) data enables the market researcher to 
measure the number of respondents who think, feel or behave in particular 
ways in relation to a wide range of issues, and to compare the responses that 
occur in different groups of respondents. For instance, are the responses of 
those aged 18-25 similar to the whole group? Are the responses of the 
women aged 18-25 similar to those of the men aged 18-25? 

Quantitative data for market research is collected through surveys. Most 
market research surveys rely on structured questionnaires which need to be 
constructed carefully, tested and piloted if they are to yield good results. 

SURVEY C^E™0©S 

There are three main survey methods: mail, telephone, and face to face. 
For a full discussion of mail and telephone interviews see Green et al 
(pages 176-184). The table below summarises the capacity of each of the 
three methods for meeting specific needs of a particular research project. 
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Important aspect of survey Performance capacity 





Mail 


Telephone 


Face to face 


Complex questionnaire 


Poor 


Fair 


Excellent 


Long questionnaire 


Fair 


Poor 


Excellent 


Geographic spread 


Good 


Fair 


Poor 


Sample control 


Poor 


Good 


Excellent 


Response rate 


Poor 


Good 


Fair 


Respondent cooperation 


Poor to fair 


Good 


Excellent 


Ease of call-back 


Poor 


Excellent 


Fair 


Accuracy for sensitive questions 


Good 


Fair 


Poor to fair 


Control over interviewer effects 


Excellent 


Good 


Poor 



GVOsdOD scLotr^e^s 

Mail surveys require respondents to complete the questionnaire themselves. 
The anonymity and privacy of a mail survey is the most effective way to 
elicit accurate and open responses to sensitive questions. A further 
advantage of a mail survey is that there is no interviewer to bias the results. 
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Every respondent is responding to the same set of stimuli presented in 
exactly the same way. However, the questions need to be unbiased and clear. 

It is wise to avoid a complex questionnaire in a mail survey. Respondents 
tend to become confused or to misunderstand the questions when grappling 
alone with a complex questionnaire. 

Questionnaires for mail surveys should be kept as short as possible to 
optimise the response rate, especially as mail surveys tend to elicit a much 
lower response rate than other survey methods. The response rate, which 
can be as low as 5 per cent, depends on the nature and length of the 
questionnaire, the level of involvement of the participants with the topic 
surveyed, the ease of the process by which they can return the completed 
questionnaire to you, and conscientious follow-up. You may expect the best 
response rate from target groups who are well known to you and for whom 
the topic being researched is of vital concern. 

Electronic mail surveys conducted by fax or by email have considerable 
advantages over traditional postal mail in that they can be more immediate 
and convenient for some respondents. They are likely to be used more 
frequently as more people obtain access to these technologies. 

A major advantage of a mail survey over telephone or face-to-face surveys 
is its lower cost. However, the considerable costs of developing the 
questionnaire and of stationery and postage demand a reasonable response 
rate to make the exercise worthwhile. (Note: Instead of sending out stamped 
self-addressed envelopes to all potential respondents, contain the costs of 
a mail survey by using the FreePost system whereby postage is paid only 
on responses that are returned to you). 

In-class surveys 

In-class completion is an efficient way to have respondents fill in a survey. 
However, there may be some bias because the respondents may be concerned 
about the level of anonymity offered. Make sure you stress that the results 
are anonymous and will be analysed only after collation. Do not ask 
respondents to put their name on the questionnaire. 

ToOofpthoim© 

Telephone interviews are regularly used in market research today. They have 
advantages over mail surveys in that the information can be collected 
quickly and with a lower rate of refusal. People are more likely to agree to 
answer questions on the telephone than in a mail survey because they can 
respond immediately and with less effort. 



From the researcher’s point of view, telephone surveys can be conducted 
very quickly, there is no need for travel, and no delay in obtaining results. 
It can be very simple to phone back at a time that is more convenient if the 
respondent is unable, initially, to comply with the request for the interview. 

Fa c © -tica> -Ira) c© sciao"^©^© 

Face-to-face interviews provide a lot of flexibility in the interviewing 
process. They are an excellent way of collecting large amounts of 
information using a complex questionnaire. They allow for interaction 
between the interviewer and the respondent. Materials such as lists to 
choose from, pictures to remind and so on, can be used as prompts. Non- 
verbal cues of confusion or reluctance can be observed and accommodated. 

However, the interviewer can introduce distortion or bias by not 
administering the survey in exactly the same way for each respondent. 
Interviewers need to be trained to read the questions exactly as written with 
an even intonation, and not to show physical reactions such as surprise or 
disagreement to responses given. 

Face-to-face surveys tend to cost more than mail or telephone surveys 
because they employ more interviewer time and usually require travel by 
the interviewer. 

The reliability and validity of survey results are dependent on the way that 
every aspect of the survey is planned. In the case of a questionnaire survey 
- whether by mail, telephone or face-to-face - the questions asked of the 
respondents need to be constructed with great care (for a full discussion on 
questionnaire design, see Alreck & Settle). 

©1? (^][lfl©S'Go<n>Q’'DS 

To be effective, survey questions should have three characteristics: focus, 
brevity and clarity. 

Focus 

Every question should focus directly on a single, specific issue or topic - 
not as easy as it might seem. 

Wrong: Which institute do you like best? 

Right: Which of these institutes would you choose to attend? 

The researcher is attempting to measure student preference. By asking 
which institute the student likes best, you may find out which they prefer 
but it may not be the institute they would choose. They may like an institute 






that is a long way from home or one where the entrance requirements are 
too high for them. The second question focuses on which of the institutes 
they would choose. 

Wrong: When do you usually go to the institute? 

Right: What time do you usually get to the institute? 

Here the researcher wants to know what time the student ordinarily goes 
to the institute. The student could well respond to the first question with 
days of the week, term dates or times. The second question focuses on the 
specific information required. 

Brevity 

The longer the question the harder it is to answer it, and therefore the higher 
the possibility of error. Respondents are likely to forget the first part of a 
long question by the time they read or hear the last part. The greater 
efficiency of the right version is apparent in the two examples below. 

Wrong: Can you tell me how many children you have, whether they are 

girls or boys, and how old they are? 

Right: What is the age and sex of each of your children? 

Wrong: If you have done one or more subjects in your course, please list 

the year and your result in each one, starting with the first subject. 

Right: Please list the year and result of each subject you have done in 

this course. 

Clarity 

The meaning of the question must be clear to every respondent, a difficult 
task that requires avoidance of all ambiguity. 

Wrong: What do you have to say about the institute that you go to? 

Right: How much influence do you personally have on which institute 

you attend? 

The first question can be interpreted as asking for the respondent’s 
opinion of the institute, rather than about their influence on deciding which 
institute to attend. 

Wrong: Ordinarily, do you take aspirin when you feel some discomfort 

or when you feel actual pain? 

Right: Do you usually take aspirin as soon as you feel some discomfort, 

or only when you feel actual pain? 
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Here the researcher wants to know whether the respondent takes an 
aspirin either at the first sign of discomfort or else waits until there is the 
perception of actual pain. However, respondents might well reply to the first 
question with, ‘Yes I do’. The researcher would not learn at what point the 
aspirin was actually taken. 



Focmatt off (maaesttooms 

Two kinds of questions are used in a questionnaire survey - open-ended 
questions and closed questions. Some questionnaires combine open-ended 
and closed questions. 
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Open-ended questions 

Open-ended questions have no predetermined response schedule. They are 
used for eliciting a wide range of responses. However, open-ended questions, 
while giving more information, are expensive to use in a questionnaire 
because they take longer to ask and to answer and because they are more 
difficult to code and analyse. Recording verbatim responses in the field is 
difficult and time consuming. There is a high likelihood of error. 

Responses to open-ended questions are not directly comparable with one 
another. They have to be placed into categories for editing and postcoding 
stages - a time-consuming and laborious task. Additionally, the analyst has 
to judge the meaning of the responses. 

Use of open-ended questions in questionnaires should be limited. 
Exploratory information is better obtained through qualitative data that 
contributes to the questionnaire design. This is especially the case if you 
are going to analyse the survey results internally. 

If you are going to use open-ended questions decide at questionnaire 
design stage how you are going to analyse them. 

Closed questions 

Closed questions are commonly used in questionnaires. They provide a 
predetermined list of possible answers for each question, from which 
respondents choose the answer that best suits their view or experience. 
Closed questions make for manageable analysis of the responses. Because 
some respondents may feel that none of the answers best reflects their view 
or experience, include a category for the respondent to nominate an ‘Other’ 
response or ‘Don’t know’. 

Types of closed questions 

The following seven types of closed questions are commonly used in 
surveys. In some surveys, a combination of types is used. 
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Yes/no options 



Yes 

No 



Have you ever been to a training course? 



Multiple choice 

1 Respondents must choose one response from a given set 

Which one of the following is the most likely to increase the skills of a manager? 
(Tick one only.) 

A) On the job training 

B) Workshops on specific issues 

C) University training 

D) Previous management experience 



2 Respondents are able to make more than one response from a given set 

What in-service training have you participated in this year? 

(You may tick more than one.) 

A) General introduction to union issues 

B) Program evaluation 

C) Management training 

D) Improving organisational effectiveness 

E) Positively coping with stress 



Likert scale 

Respondents are asked to rate the extent to which they agree or disagree with 
a statement 

The institute should provide the option of classes on Saturdays and Sundays. 
(Circle the column that best matches your view.) 



Strongly 

agree 


Agree 


Not sure 


Disagree 


Strongly 

disagree 


1 


2 


3 


4 


5 



Guttman scale 

Respondents must select from a scale which is cumulative 

(that is, if a respondent has ticked B, it implies that they have achieved A) 

What is the highest level of education you have achieved? 

(Tick one only) 

Primary school 
Secondary school 
Tertiary institution 
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Semantic differential 



Respondents are asked to rate something at any point along a continuum with 
opposed adjectives at either end 

Rate the training workshop that you participated in today along the following 
dimensions. If you found the workshop very useful, you should circle a high number. 
If you found the workshop useless you should circle a low number. If you did not find 
it particularly useful or useless, you should circle a number in between. The same 
applies to uninteresting/interesting below. 



Useless 




Neither useful not useless 




Useful 


1 2 


3 


4 5 6 7 


8 


9 10 



Uninteresting 












Interesting 


1 2 3 


4 


5 


6 


7 


8 


9 10 



The ten-point scale used in this semantic differential may create some problems in 
analysis. What constitutes a good result on a ten-point scale? There is no simple and 
absolute answer to that. With one group of respondents, 7 out of 10 might be 
considered a top score while, with another group, 7 out of 1 0 might be only average. 
These scales need to be analysed very carefully within the context of the overall 
responses of the sample. 



Rank order 

Respondents are asked to put items in order of importance 

Please number the following health problems in the community in order of importance. 
Place a 1 next to the problem which you consider most important. Place a 2 next to 
the problem which you think is next in importance and so on through to 4 for the 
problem of least importance. 

Alcohol and drug dependence 

Loneliness and isolation 

Motor vehicle accidents 

Pollution 

While rank ordering is a popular form of scaling for researchers, if respondents 
are asked to rank order more than five items, they fail to make real discriminations 
between items. 



56 




Questions in a survey must be expressed in language that is simple and 
unambiguous. Both vocabulary and grammar are important. A good 
discussion of question form and question wording can be found in Schuman 
& Presser and in Sudman & Bradman. 



Vocabulary 

The vocabulary has to be understood by the least sophisticated of your 
respondents. Don’t presume too much. Everybody has a core vocabulary that 
consists of words that they use in everyday speech, and a wider vocabulary 
of words that they recognise and understand but don’t use commonly. And 
there are words that they do not know at all. 

As a rule of thumb the researcher should always use words that are in 
the core vocabulary of the respondents. The following (rather extreme) 
example demonstrates the effectiveness and clarity of simple language. 

Wrong: What emotions were evoked by perceiving the spectacle? 

Right: What kinds of feelings did you have when you saw it all? 

Be wary of using technical terms which you may use all the time but which 
respondents may not be familiar with. For example, don’t ask a student about 
how many ‘core courses’ they are doing unless you are sure that they will 
understand what you mean by that term. 

Grammar 

Sentence structure is important in ensuring that your survey is clear and 
unambiguous. Try to use a simple sentence with just a subject, a verb and a 
predicate. Try to avoid compound sentences. Change long dependent clauses 
into single words or phrases wherever possible. 

Wrong: What would you do when you had only a few things to buy and 

there were a lot of people in the checkout line? 

Right: Suppose you only have a few things to buy. There are a lot of 

people in the checkout line. What would you do? 

Wrong: If you didn’t have a reservation ahead of time and you found out 

that the only seats that you could get were at the very top of the 
upper balcony, what would you do? 

Right: What would you do if the only seats available were at the top of 

the upper balcony? 
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Firamnirog ttlhe cgcmesiiootrDs 

Error and bias can be introduced into the research through poorly framed 
questions. If the questions themselves cause error in the response, such 
systematic bias will seriously affect your results. Five commonly found 
errors are described below. 
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Ambiguity 

In the first question below there is no indication to the respondent of whose 
point of view they should take in framing their response. Consequently, 
some respondents may state their own preferences while others may state 
what they consider to be the general view. The correct question indicates 
that the unit of analysis is to be the respondent themselves. 

Wrong: How important is it for institutes to provide a large number of 

different types of courses? 

Right: How important is it to you that the institute you attend provides 

a large number of different types of courses? 

Emotive language 

Avoid vocabulary that might bias the response, especially emotive adjectives. 
Use non-emotive, neutral terms. The example below shows how just one 
word can make a big difference in the meaning conveyed. 

Wrong: Would you favour increasing course fees to cope with the current 

education crisis ? 

Right: Would you favour increasing course fees to cope with the current 

education problem ? 

Double-barrelled questions 

A common problem in poorly constructed surveys is to ask two questions 
in one. A respondent cannot give a single answer to a double-barrelled 
question. The solution is to ask two single questions. It also takes less time 
to answer two simple, clear questions than it does to answer one poorly 
constructed question. 

Wrong: What do you think of the quality of the food and service in the 

canteen? 

Right: What do you think of the quality of the food in the canteen? 

What do you think of the quality of the service in the canteen? 

Leading questions 

Leading questions predispose the respondent to answer in a particular way 
and seriously bias the results. 

Wrong: Don’t you feel the institute should introduce the new policy? 

Right: Do you feel that the institute should introduce the new policy? 
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Loaded questions 

A loaded question is a subtle form of a leading question. The loaded question 
includes some wording or phrasing that is designed to influence and sway 
a response. In the example below, the phrase ‘an excellent outcome’ suggests 
a very desirable goal. It would be difficult to disagree with this sentiment 
without appearing not to value excellence as an outcome. 

Wrong: Do you advocate longer contact hours to ensure an excellent 

outcome in this course? 



Bight: 



Does this course require more contact hours? 



COMMON PITFALLS IN WRITING QUESTIONS 



Pitfalls 



Examples 



Poor choice 
of vocabulary 



Wrong 

Are you cognisant of all of the concepts to be 
elucidated? 

Right 

Do you know about all the ideas that will be 
explained? 



Poor grammar Wrong 

What would you do when you had only a few things 
to buy and there were a lot of people in the 
checkout line? 

Right 

Suppose you only have a few things to buy. There 
are a lot of people in the checkout line. What would 
you do? 



Ambiguity Wrong 

How important is it for institutes to provide a large 
number of different types of courses? 

Right 

How important is it to you that the institute that you 
attend provides a large number of different types of 
courses? 



Emotive language Wrong 

Would you favour increasing course fees to cope 
with the current education crisis? 



Right 

Would you favour increasing course fees to cope 
with the current education problem? 
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Double-barrelled 

questions 



Wrong 

What do you think of the quality of the food and 
service in the canteen? 

Right 

What do you think of the quality of the food in the 
canteen? What do you think of the quality of the 
service in the canteen? 



Leading questions Wrong 



Don’t you feel the institute should introduce 
the new policy? 

Right 

Do you feel that the institute should introduce the 
new policy? 



Loaded questions Wrong 



Do you advocate longer contact hours to ensure an 
excellent outcome in this course? 

Right 

Does this course require more contact hours? 



QUESTIONNAIRE DESIGN 

1 Focus precisely. Every item should focus on one specific issue or topic. 

2 Keep each item brief. The longer the question, the greater the possibility of 
error and bias. 

3 Strive for clarity. Every respondent must know exactly what is being asked. 

4 Use core vocabulary. Use the same words as the least sophisticated 
respondent would use in common speech. 

5 Use simple sentences. Two or more simple sentences are preferable to 
one compound sentence. 

6 Avoid sources of bias and error. Be sure items are free from the factors 
that create bias and error. 

7 Use structured questions. Unstructured questions usually generate large 
amounts of poor-quality data. 

8 Avoid response bias by carefully reviewing questions, the order of 
questions and the structure of the questionnaire. 
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Overall organisation 

Introducing the questionnaire 

Most questionnaires rely on the cooperation of respondents. Make sure that 
the questionnaire survey is introduced in a manner that makes it appealing 
to the potential respondent. A good introduction increases the response rate 
and positions the research as credible and worthwhile. 

Initial questions 

The questionnaire should begin with questions that are non-threatening and 
that ‘set the stage’ for the questions to follow. These initial questions should 
also be relatively simple, quick and easy to answer and generally applicable 
to all respondents. 

Body of the questionnaire 

This is where questions relating to the major issues in the research are asked. 
The complex and detailed questions are asked in this section. Make sure that 
the questions follow a logical order or flow and do not seem to the 
respondent to jump all over the place. 

Conclusion 

The final section of the questionnaire may be used to ask the more sensitive 
questions and demographic questions. These questions are generally asked 
last because by that time the respondent should be feeling comfortable about 
answering questions and be more likely to be drawn on a sensitive topic. 
Alternatively, if the respondent refuses to continue with the questionnaire 
you already have most of the responses that you need. 

Grouping the questions 

Within each of the above three sections, specific questions (or items, as they 
are often called) are grouped within subsections. A full questionnaire will 
often have 100 or more individual items in it. 

A well constructed questionnaire should appear to the respondent as a 
logical, integrated and consistent ‘exchange’ between the researcher and 
themselves. When you are designing the final format of the questionnaire 
think about the way the respondent will perceive the questions and try to 
make the flow and style of the questions meaningful and credible to the 
respondent. 

By topic 

Questions relating to the same topic may be grouped together. For example, 
a questionnaire about institute preferences may contain questions about 
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three different topics: the general attitudes of the respondents to post- 
secondary education; specific attitudes to specific institutes; and actual 
behaviour in choosing an institute. Items relating to each of the topics would 
be grouped together. 

By scaling technique 

If a number of survey questions are to be answered using the same scaling 
technique, then it may make sense to group these questions in one section 
of the questionnaire. For example, all the questions using a Likert scale 
(Strongly agree, Agree, Disagree, or Strongly disagree) could be grouped 
together. This makes the response task easier because the respondent has 
to learn to use that scale only once. 

Branching or ‘skip’ questions 

In most questionnaires there will be some questions that will not apply to 
all respondents. In such cases you need to include a ‘skip’ instruction. For 
example, you may want to ask specific questions of respondents who had 
heard about your institute through a newspaper advertisement. The 
questionnaire must establish whether or not the respondent did hear about 
the institute in this way. If so, the specific question(s) will be asked but, if 
not, respondents would not be asked the question(s) but told to move on to 
another item. This process is demonstrated in the example below. 
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USE OF ‘SKIP’ OR ‘GO TO’ INSTRUCTIONS 



Did you hear about this institute through 
a newspaper advertisement? 


Yes No 


[If you answer No, go to Question 5] 





Did you hear about this institute through 
a newspaper advertisement? 


Yes No 


[If you answer No, skip Question 4] 





How did you hear 


Newspaper 


[Go to Question 5] 


about this institute? 


Television 


[Go to Question 6] 




Friends 


[Go to Question 7] 




Teachers 


[Go to Question 8] 



The ‘go to’ instruction is ordinarily used when it is necessary for the 
respondent to pass over several items. The ‘skip’ instruction is used when 
only one or two items need to be passed over. The questions to be ‘skipped’ 




should immediately follow the instruction, otherwise it will be too confusing 
for the respondent or for the interviewer. It is not a good idea to use this 
form of ‘skipping’ in a self-administered questionnaire but it can be used 
in interview surveys. 

Instructions 

All questionnaires require the inclusion of some instructions to let the 
interviewer or the respondent know how they are to ask or respond to a 
question. Instructions will generally apply to a set of items. 

Scaling instructions 

Scaling instructions should be clear, concise and unambiguous. The first set 
of instructions below does not indicate how many of the items should be 
rated, is unclear about whether the items should be ranked or rated, and does 
not indicate where the response is to be recorded. Although you might think 
that all these things would be obvious to most respondents, it is important 
to present the instructions in the clearest possible way to minimise potential 
misunderstanding and error. 



Wrong 



Please rate the importance of the following. 




Scale 

Unimportant 1 2 3 4 5 


Important 


Library services 

Cafeteria facilities 

Course options 

Timetable flexibility 




Right 


Please rate how important each of these items is to you. If it is not at all important 
to you, choose number 1 . If it is very important to you choose the number 5. If it 
is somewhat important to you, choose a number from the middle of the scale. 
Write the number you choose in the space in front of the item. 


Scale 

Unimportant 1 2 3 4 5 


Important 


Library services 

Cafeteria facilities 

Course options 

Timetable flexibility 

Interviewer instructions 
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Interviewer Instructions 

When an interviewer is administering a survey questionnaire to 
respondents either by telephone or face to face, there are many occasions 
when it is necessary to include instructions to the interviewer. 

Interviewer instructions should be clearly distinguished from the 
questions to be asked of respondents. Instructions to the interviewer should 
be enclosed in parentheses or written in upper case letters or italics, to 
indicate their status. 



Q20 

I’d like to get some thoughts from you on how the institute can increase its 
enrolments. Which of the following do you think the institute should focus on to 
increase enrolment levels? Rotate the starting point between respondents. Circle 



each one nominated. Should it focus on: 

Changing the course options in some way? 1 

Lowering the cost of the courses? 2 

Targeting different types of participants? 3 

Promoting the institute more heavily? 4 

Other suggestion which is: 5 
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Piloting 

The final step in questionnaire construction is to pilot the survey, to test it 
with a small number of respondents. Depending on the size of the study and 
the range of respondent types, a pilot of 10 to 30 people is usually sufficient 
(see Hunt et al for a full discussion of the issues involved in piloting a 
questionnaire). 

Respondents should be told that the questionnaire is a pilot, and that they 
are free to comment on anything which they find confusing or unclear or 
to ask questions if they need more information. Interviewers should also be 
observant, noting any difficulty experienced by respondents in completing 
the questionnaire. 

Piloting the questionnaire is also an excellent way of timing how long the 
questionnaire takes to complete. If you are paying someone else to collect 
the data, this can be very important, as you will know more about the cost 
of collecting the information. 

You should never use a questionnaire that has not been thoroughly and 
fully piloted. 




MAXIMISING RESPONSE RATES TO THE QUESTIONNAIRE 



1 Is the questionnaire easy to complete? 

2 Does the first page have a simple introduction explaining the purpose of 
the research, the importance of taking part and assuring confidentiality? 

3 Is the wording used straightforward and expressed in everyday language? 

4 Does the first page of the questionnaire contain only easy non-threatening 
questions? It is better to start off with simple, factual questions rather than 
complex, opinion-based questions. 

5 Are the more complex questions reserved for the main body of the 
questionnaire? Do they proceed from the general to the more specific in an 
logical, integrated manner? 

6 Are more sensitive questions left until last? Sensitive and demographic 
questions should be left until towards the end of the questionnaire. (Some 
people regard questions relating to their age, income and education as 
very personal.) 

7 Are the pages attached so that they will not become separated or lost? 

8 Are all the pages clearly numbered and arranged so that the interviewers or 
the respondents can follow the sequence easily? 

9 Does each page have a note at the bottom directing interviewers/ 
respondents to the next page? 

10 Do the sections within the questionnaire form simple steps or subtasks 
which can be completed one at a time? 

1 1 Does the questionnaire have ample white space, to avoid a dense cramped 
or cluttered appearance? 

1 2 Has the questionnaire been piloted? 



Refer to Weiers (pages 232-239) for a full discussion of techniques to 
maximise response rates to questionnaires. 



QMESYOOMAOrciE S$UM][P[LES 

Overleaf are annotated sample segments from self-completion questionnaires 
and telephone questionnaires. These samples demonstrate some commonly 
used practices in questionnaire design. However, they are not intended to 
be used as proformas. 
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1 TAFE STUDENT QUESTIONNAIRE FOR SELF-COMPLETION 
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Instructions to respondents 

Purpose of collection 

This national survey is collecting information from all 199x TAFE 
graduates. The survey will assist in measuring the transition of 
TAFE graduates into the labour market, and improving the quality 
and relevance of TAFE courses. 

Confidentiality 

Your completed form remains confidential to the researchers. 

No information will be released in a way that would enable 
identification of any individual. 

Due date 

Please complete the form and return it in the reply-paid envelope to 
X organisation by 30 June 199x. 

Help available 

If you have any problems in completing this form please contact us: 
Telephone 1 800 XXXXXX Facsimile (03) XXXXXXX 

How to answer 

Please tick one box only (unless otherwise requested). Some tick 
boxes have ‘Go to’ instructions. By following the ‘Go to’ you skip 
questions that are not relevant to you. 



Instructions 

to 

respondent 
printed in 
upper case 
and bold 



Precoding 
of answers 
indicated 



Open-ended 

question 



Q1 .1 Why are you doing this course? 

TICK ALL THE ANSWERS THAT APPLY TO YOU IN THE 
FIRST COLUMN. YOU CAN TICK MORE THAN ONE BOX. 

□ i No jobs available _ 

□ 2 Compulsory part of apprenticeship 

□ 3 To get a qualification 

□ 4 To gain industry skills 

□ 5 To get a job 

□ e To upgrade my job skills 

□ 7 For general interest - not for a job 

□ a As a stepping stone into another course 

□ 9 Because the CES makes it compulsory 

□ 10 To help get promotion or a better job 

□ 11 To fill in time 

□ 12 My parents want me to 

□ 13 I could get in with my school results 

□ 14 It’s exactly the course for the career I want 

□ 15 It’s not too big a step for me at this time 

□ is My teacher’s advice 

□ 17 I couldn’t get into university _ 

□ is Some other reason _ 




Q1 .2 Now think about the two reasons that were most 
important to you in deciding to do this course. 

PUT A T IN THE FINAL COLUMN ON THE LINE NEXT TO 
THE REASON WHICH IS THE MOST IMPORTANT FOR YOU. 
THEN PUT A ‘2’ NEXT TO THE SECOND MOST IMPORTANT 
REASON. 



Underlining 
used to 
emphasise 
important 
terms or 
phrases 



Q2 Did you have any full time, part-time or casual 
job of any kind (including an apprenticeship), 
that lasted for two weeks or more, during the 
six months before starting your course? 



Note: 

If you had more than one job, please report your 
main job - the job in which you usually worked the 
most hours. 



Yes, worked for payment or 
Yes, but on leave from main 
Yes, unpaid work in a family 
Yes, other unpaid work 
No, did not have a job 



profit 


□ 


GO TO Q4 


job 


□ 


GO TO Q4 


business 


□ 


GO TO Q4 



Q3 Did you actively look for work during the 
six months before starting the course? 



Notes 
included 
to clarify 
meaning 



Skip 

question 



Note: 

Examples of actively looking for work include: 
being registered with the CES; writing, telephoning or 
applying in person to an employer for work; or 
advertising for work. 



Yes, looked for full-time work 
Yes, looked for part-time work 
No, did not look for work 



□ G0T0Q6 

□ G0T0Q6 

□ G0T0Q6 
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Rotation 
instruction - 
to avoid bias 
in responses 



Q1 a Thinking back to how you found out about this 
program, from which of the following sources 
did you hear about the program? 

ROTATE START POINT BETWEEN RESPONDENTS 

Q1 b And what was the main single source which 
prompted you to inquire about the program? 



Advertising eg newspaper, magazine or radio 
Articles in newspapers, radio, TV 
Via past participants 

Open-ended 

question Via direct mail 

Via your general business contacts 
A Via another source (please specify) 



Qla Qlb 

All Main 

1 1 

2 2 

3 3 

4 4 

5 5 

6 6 



Don’t know/can't remember 



7 7 
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Q2 

Note the 
conversational 
tone 



Key points 
underlined to 
ensure clarity 
in insructions 



We are interested in what motivated you to 
take part in the program. I am going to read 
you a number of possible reasons for doing 
the program, and I would like you to give me a 
score from zero to ten to show how important 
each factor was to you. If it was very 
important , give a score of 1 0, and if it wasn’t 
important at all , give a score of zero. You can 
give any score from zero to ten to show how 
you feel. 



Check 

instruction 

was 

understood 



ROTATE START POINT BETWEEN RESPONDENTS. AFTER 
FIRST SCORE, CHECK RESPONDENT HAS INTERPRETED 
SCALE CORRECTLY eg, IF SCORE OF 8 GIVEN, SAY: ‘SO 
THAT WAS IMPORTANT TO YOU’. 

Here’s the first statement. 



Coding 



Instruction to 
interviewer 
not to 
paraphrase 
response but 
to record 
what was 
said 



I took part in the program: 

to develop my general business management skills □ i 



to learn some specific business skills ds 

to develop entrepreneurial skills da 

to build a network of business contacts - d 4 

to obtain credits towards a tertiary qualification ds 
to meet professional development criteria de 

to improve my career prosoects d? 

for some other reason, which was de 

* (RECORD VERBATIM) 




BEST copy AVAILABLE 



Q3 Thinking about the main reasons for your 
undertaking the program, how well did the 
course meet your needs? Did it meet your 



needs: 

Very well d i 

Quite well d? 

No real option □ 3 

Quite poorly d< 

Very poorly □ 5 

Hard to say de 



IF RESPONSE WAS ‘VERY WELL’, ‘QUITE WELL’, ‘QUITE POORLY’ 
OR ‘VERY POORLY’, ASK Q4, OTHERWISE GO TO Q5. < 



A range of 
descriptors, 
so as to 
pick up most 
responses 



Skip question 
directs 
interviewer 
to one of 
two options 



Q4 Why do you say that? 

FURTHER PROMPT: Anything else? 



Instructions 
to interviewer 
to probe 
further for 
responses 



Q5 When you were considering this program, did 
you consider any other program as well? 

Yes d 1 

No d2 

Don’t know/Can’t remember d^ <" 



Option for 
those 

respondents 
who can't 
remember 



ER | C ST COPY AVAILABLE 



Demographics are often placed at the end of an administered 
questionnaire because by that time in the process most respondents are 
sufficiently comfortable with the process to be willing to volunteer such 
information. Questionnaires by the Australian Bureau of Statistics and other 
bodies who can collect information under the authority of the Census and 
Statistics Act sometimes begin with demographic questions. However, if you 
are relying entirely on the respondent’s goodwill to collect the information, 
it is advisable to place the demographics at the end of the questionnaire. 



3 DEMOGRAPHICS SEGMENT OF A FACE-TO-FACE OR 
TELEPHONE INTERVIEW 



Use 

categories 
rather than 
ask for a 
specific age 



Option for 
those 

unwilling to 
disclose 



I now have a few final questions which will help us classify the 
information we collect. 



Q1 Which one of the following age groups do you 

belong to? Are you ...? 


•> Under 25 


□ - 


25-39 


□ a 


40-50 


□ a 


Q2 Which of the following best describes the 

annual pre-tax income of your household? 


Less than $10,000 


□ 1 


$10,000-$29,999 


□ 2 


$30,000-$49,999 


□ 3 


$50,000-$79,000 


□ « 


$80,000 plus 


□ a 


Not stated 





Q3 Which of the following best describes your 
final level of education? 



Completed high school to Year 10 □ i 

Completed high school to Year 12 □ 2 

Post-secondary qualification (certificate/diploma) CU 
University degree (3 year degree) CU 

Graduate diploma CU 



re 



BEST COPY AVAILABLE 





Q4 In which country were you born? 


Time-saving 


ALLOW RESPONDENT TO ANSWER UNPROMPTED AND THEN 


and clearer 


ALLOCATE ANSWER TO ONE OF THE CODES 




not to read 
through 
all the 
categories. 


Australia 


□« 


UK or New Zealand 


□2 


This is not 


Vietnam 




generally a 
sensitive 


China 


□< 


question 


Hong Kong 


□5 




Philippines 


□« 




Indonesia 


□7 




Other (please specify below) 


□ a 





If a 


Q5 Record sex 

Female 

Male 


□> 

□2 


itJiwpnunw 

interview, 
question 5 
and 6 are 
answered 
without 






reference 


Q6 Record state 




to the 






respondent 


NSW 


□1 




Victoria 


□2 




South Australia 


□3 




Queensland 


□< 
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[FOUL© work 

Conducting field work requires understanding and some organisation of the 
process. Interviewers should be selected with great care. Not everyone makes 
a good interviewer. In qualitative research the calibre of interviewers is as 
important as the discussion guide. In quantitative research the capability 
of the interviewer is still very important. 

Oc'DttetTOooweo' ft tr soon mg] 

Interviewer Quality Control Australia has set out a code of practice for field 
work in all areas of market research in Australia. Its Accreditation Manual 
sets out detailed requirements for meeting standards of industry practice in 
interviewing. It is fully endorsed by the Market Research Society of 
Australia, the Association of Market Research Organisations, and the 
Field Management Group. The Executive Director of Interviewer 
Quality Control Australia can be contacted through Market Research Society 
of Australia (Victoria), PO Box 582D, GPO Melbourne Victoria 3001 
Telephone (03) 9499 2679. 

Interviewer Quality Control Australia stipulates that all interviewers 
require a minimum of 4 hours of classroom training in the following areas: 

Definition of market research 
Code of conduct in relation to interviewing 
Approaching the respondent 

Identification procedures to establish bona fides of the interviewer/ 
company 

Coping with refusals 
Conducting an interview using: 

- closed questions 

- open-ended questions 

- scales 

- administration and completion of questionnaires 

- field sampling procedures and quota control 

- terminology and jargon 
Completing an interview 

Quality control procedures applied to interviewers. 

There are many guides to training interviewers available. Commence with 
the Fieldwork Handbook produced by the Market Research Society of 
Australia and recommended by Interviewer Quality Control Australia. 






Fotr [r©s©si[r©[h) 

Qualitative research is highly dependent on the calibre of the interviewers 
or group moderators. As the interviewers use the questions as a guide rather 
than as strict instructions, qualitative interviewing requires a very highly 
trained and skilled field force. Selection of interviewers is critical. They 
need to use their initiative in deciding which issues to follow up and probe, 
and which ones to let go. 

Briefing of interviewers should provide them with full background to the 
project, the research objectives, the type of information that needs to be 
collected and how the research will be used. The more that qualitative 
interviewers understand about the nature of the project, the better equipped 
they will be to guide the respondent to discuss the most relevant matters - 
even if they are not on the discussion guide. 

Check and supervise the work as it comes in. Do not wait until the end 
of the survey to find that one of your interviewers is asking or recording 
something incorrectly. Quality control is essential. Qualitative interviewing 
often benefits from a preliminary debriefing half way through the project 
to check how the research is going. This debriefing may also identify issues 
raised by some respondents that merit exploration with other respondents. 

Fotr cgoaanniiioiisatiS^© x<b seatrcih] 

Full briefing of the interviewers and in-house practice in using the 
questionnaire is recommended. Even a little training improves the quality 
of the data which you will collect. Never hand a questionnaire to someone 
and tell them to go out and use it. 

With a structured questionnaire, stress to the interviewers that they must: 

• ask the questions and present set answers exactly as they are written so 
that they remain as free as possible from bias and error; 

• not add information that is not in the questionnaire. The interviewer 
should not try to ‘help out’ by providing ad hoc explanations of the 
questions; 

• not accept instant ‘don’t knows’. 

Data processing, like all other aspects of the research process, can be 
outsourced to an agency. Even if you are conducting most of the research 
yourself, there are advantages to outsourcing the data processing for a large 
quantitative survey. 
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Get a head start on the process without waiting for all data to arrive - while 
the survey is being conducted or even earlier. 

Expect a large volume of paperwork. Surveys often generate more material 
than anticipated. Keep the process neat and orderly. Have a place for 
everything and everything in its place. It is a good idea to number each 
questionnaire so that it is identifiable for later checking. 

The first task is to proof read the questionnaires to make sure that they have 
been completed correctly and fully. It is standard industry practice to ‘audit’ 
10 per cent of the total sample to make sure that they were completed by the 
respondents nominated. All accredited agencies follow this procedure. If you 
are conducting your own research it is a good practice to follow. 

If you are having someone else in the institute enter and process the data, 
make sure that you have discussed and reached understanding on their and 
your requirements for the process. Explain to them the criteria for acceptance 
and rejection of completed questionnaires and make sure that these criteria 
are used consistently. Ask them to inform you how the final data sheets will 
be presented. Discuss any changes you want to that system of presentation 
before processing starts. 

The overall task of processing basic survey data is generally ‘easy’ because 
it is ‘packaged’. The raw data is keyed into a computer and an appropriate 
software package used to collate the data in the ways that you and the 
processor have determined. If you are undertaking complex, multivariate 
analysis (such as factor analysis, cluster analysis, segmentation, choice 
modelling and so on) requiring expert processing, make sure that you and 
your ‘expert’ both fully understand what can be done with the data and what 
is required of the processing. 

If you have more than about 50 respondents and about 30 questions, don’t 
try to analyse your own data using a spreadsheet application. Spreadsheets 
are not designed for such a purpose. Instead use an analysis package such 
as SPSS, Statview or Microtab which is designed to provide a range of 
outcomes from basic frequencies and cross-tabs through to more 
sophisticated analytical techniques such as regression analysis and 
perceptual mapping. 

If you need to use an analysis package and you can afford it, use a 
processing agency, whether external or in-house, that is accustomed to this 
type of processing. They can probably complete the job more quickly and 
cheaply than you can yourself. 

Detailed mathematical discussions of data processing can be found in Dillon 
et al (chapters 13-16); Weiers (chapters 12-16) and Green (chapters 15-20). 





There is no one best way to process qualitative data. Each qualitative 
researcher will have a preferred method. However, there are common stages 
in the process. 

Preparation of data 

Full and detailed notes are needed on every interview. Each interviewer 
should provide you with typed notes on each interview conducted. 

The debriefing 

Debriefing a qualitative field force is a very important part of the processing 
(and analysis). It can take several hours to run through the topics outlined 
in the discussion guide. It is a good idea to tape record the debriefing to make 
sure that you retain all the views of the research team. The debriefing is a 
good way for you to get an overview of the differences and similarities in 
the data collected by different interviewers. 



Processing and analysis 

In qualitative research, processing and analysis of the data are usually done 
simultaneously by reading all the notes, identifying similarities and 
differences in responses between groups in the sample, and looking for 
patterns of response amongst segments in the sample. 

Analysis of qualitative data is reliant on intuition and judgement - and 
also on meticulous and careful reading of the data. Some people like to 
manage this process by using highlighters or ‘stick it’ notes or even by cutting 
out relevant comments from each transcript. Whatever way is chosen, it 
requires patience and methodical reading and re-reading of each interview 
report or group transcript a number of times, noting what is deemed 
important and relevant. 

Processing a sample of 40— 50 depth interviews is a long and exacting task. 
Allow enough time to do it thoroughly. See Gordon & Langmaid (chapter 10) 
for a further discussion of processing and analysis of qualitative data. 

A DVD A Of S 0 GVQ <3 

Analysis of qualitative data is carried out simultaneously with its processing 
(see above). Once quantitative data has been processed it is in a form that 
is ready for analysis. If you have done the processing yourself, you will have 
had some advance indications of the results. Otherwise this is your first 
contact with the raw results of the research. 

First, recall the objectives of the research. Refresh your mind with why 
you have done this research in the first place, to ensure that the statistical 
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analyses to be used are designed to suppress detail and reveal important 
findings. There is no point in running the data through every statistical test 
possible simply because you have that capability. You want to use only the 
statistical analyses that are useful to your overall aims. 

Types of analysis 

There are two types of basic analysis. The first is a frequency analysis, a 
simple count of each question to show how many respondents gave each 
answer. 

The second is one-way cross tabulations (cross tabs) to obtain numbers 
and percentages for each question, dissected according to a particular 
variable such as age, gender, educational experience or industry type. 

Key variables need to be selected for their capacity to deliver important 
findings. Is a respondent’s age likely to make a difference to the way that 
they have answered a question? Is industry background likely to influence 
the answers given? The usual variables are age, gender, education, 
occupation. There will be others that are specific to each survey, for example 
heavy/medium/light users of a course type. Specify to your processor what 
variables you want used in your data analysis. 

Once all the data is counted and cross-tabulated you can begin to analyse. 
As you analyse the data you may identify some further specific analysis 
needs, in which case you request the processor for more runs with new 
tables or scores or a more sophisticated form of analysis. 

Select the simplest tool that will deliver the results that you need. Good 
use of a simple method is far more effective than poor use of a complex 
method. 
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The research report is the means of conveying the salient results of this 
research to the decision makers who are relying on the results for their 
management decision. It is very important that the report is easy to read 
and technically correct. The suggestions below, following Alreck & Settle, 
can be used for your own report writing or can help you to judge the 
quality of a report written by an external agent. 

Foe rmatt 

All business reports, including research reports, should commence with the 
conclusions. It is important to answer the overall research questions early 
so that the reader of the report can discover quickly and easily the purposes 
for conducting the research and its findings. 

( ( 



Some busy executive staff are going to read only the executive summary 
and introduction to a full report, so those elements need to convey key 
messages and be able to stand alone if necessary. 

Executive summary 

The executive summary should be only two to three pages long and highlight 
the objectives, key findings and information sources. Keep the paragraphs 
brief- three to four sentences per paragraph. Use headings with key phrases 
to identify sections. 

The summary must be clear, concise and attract interest. A reader should 
be able to pick up the executive summary to find out quickly and easily the 
main objectives of the research and its main findings. An executive summary 
is no place for detail. It should provide a readable and informative overview 
of the research. 



Conclusions and recommendations 

The second section of the report should state the conclusions and 
recommendations that flow from the findings of the report. There should 
be a logical relationship between the findings, the conclusions drawn by the 
researcher and the recommendations that are being made. The extent to 
which recommendations are needed will depend on the research brief. In 
some cases, only findings and conclusions are called for. Again, this section 
should be concise. 

Main body of the report - the findings 

The findings should be reported as clearly as possible. Divide the text into 
sections according to types of information, topics or issues. Introduce each 
major section by identifying its purpose or the reason for including it. Avoid 
technical jargon used in statistics or processing in favour of conventional 
language. Use short simple sentences rather than long complex ones. Identify 
tables, figures and other inclusions by consecutive number. Be selective with 
content so that the main points receive emphasis. Interpret rather than 
merely describe the results. 
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GENERAL GUIDELINES FOR AN EFFECTIVE REPORT 



• Keep it simple and to the point. 

• Always refer to ‘respondents' thereby avoiding any suggestion that your results 
are applicable to the whole population. 

• Put detailed technical information in an appendix. 

• Include discussion guides and questionnaire instruments in an appendix. 

• Don’t report percentage findings to the decimal point. That gives a false 
impression of the real level of accuracy. 

• Don’t talk about small percentages of the responses as being important. 
Differences of 1 per cent and 2 per cent are often unreliable. Focus on the big 
picture. 

• Use disclaimers. If the sample size is small, say so. Always state the total 
number of respondents rather than just the percentage - if the total number of 
respondents were 6, then stating a 50 per cent preference, when referring to 
3 individuals, is misleading. 

• Use plenty of white space on the page to help convey complex and detailed 
information clearly and concisely. The report should look uncluttered and easy 
to follow. 
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The publicity and promotions officer at the institute was faced with a 
recurring problem. She had to make ongoing decisions about how to spend 
the institute’s promotional budget. This included making decisions about 
what proportion of the budget to use on advertising (compared to other 
promotional activity), what the advertising should say and where the 
advertising should be placed. 

The officer had been in the job for 2 years. The director kept stressing the 
need for effective promotions that would work ‘smarter’ with limited 
resources. Because, so far, the promotions officer had tended to make her 
decisions based on talking to people who had been at the institute for a long 
time, the promotional strategies had been minor variations on what had been 
done ‘last year’. She was not convinced they were the best strategies. 

0 itq tfoirma ft doitq cr ©<$] cm o ere m© crofts 

She came to the view that if she understood how students made their 
decisions about which institute to choose, she would have a better idea of 
what motivated the students, and who really made the decisions and at what 
points in the decision-making process. She could then direct institute 
promotions and publicity to have a real impact. But how could she find out 
some answers to these questions? 
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The promotions officer was convinced that she needed to understand the 
students’ decision processes if she was to be able to make the best decisions 
about promoting the institute. She felt a need to conduct some research with 
both current students and some potential students. 

Informal or formal research? 

Would it be enough to ask long-term staff of the institute and the students 
themselves how students decide to come to the institute? This course of 
action would take little time and money. It would not be controversial. 
Senior staff were pretty confident that they understood their students and 
their markets. They may feel that she was ignoring their expertise and 
experience by conducting formal market research. The institute had not done 
much formal research in the past and she was not sure how the senior staff 
would react to the idea. 

However, the promotions officer had been relying on internal expertise 
for the last two years and, in her view, there were serious knowledge gaps. 




The institute had been spending a significant amount of money on 
advertising, and the promotions officer had no idea what effect it was having 
or which bits were or were not working. The promotions officer decided she 
needed to do formal market research. 
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Although she was the publicity and promotions officer at the institute and 
had done some in-service marketing courses, the promotions officer knew 
very little about market research. Further, there was no-one else in the 
institute who was assigned to either marketing or to market research. 

However she had never used an external agency. How would she know 
if they were any good? Would she be able to manage them? Would the results 
be useable and worthwhile? Would she ‘blow the institute’s budget’? 

On balance, the promotions officer felt she should investigate an external 
agency. The institute did not have the available personnel resources to 
conduct its own formal research. 



Finding an agency 

As a novice in the area, the promotions officer decided to start her search 
by phoning the Market Research Society of Australia. The staff there were 
able to give her a preliminary listing of local market research agencies 
experienced in this type of research. 

She then rang a number of these agencies and discussed the task with the 
staff there. She was then able to make a short list of those agencies where 
the staff were helpful, patient and seemed to have the appropriate expertise 
in the education market. She figured that if they were not good at managing 
preliminary inquiries in a friendly and helpful way, they would not be good 
to work closely with over the duration of the project. She also rang staff in 
her position at other institutes to try to get some leads on who they used. 

Finally, she looked up the staffing lists for the institute and found that 
one of the lecturers in Business Studies had qualifications in Marketing and 
seemed to have done quite a bit of market research. As she predicted, he 
was too busy to get involved in the research. However, he was able to offer 
her some useful advice and agreed to be co-opted onto a committee for 
managing the research project. 

Tift)© [r©$©snr©ift) toiro©!? 

The promotions officer then prepared a research brief. The external agencies, 
no matter how good, could not be expected to deliver results if she was 
unsure of what she wanted from the research. The brief included a 
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descriptive introductory section, detailing the background to the project and 
the context in which it was set. It described the management decisions for 
which this research was needed, and the specific research objectives. 

The research brief also indicated the budget and timing (with an 
anticipated completion date) and requested that in its proposal the agency 
respond to particular issues, for example the research agency was asked to 
provide a detailed description of the proposed methodology and a rationale 
for its selection; and to cost each component of the methodology separately, 
so that each could be re-considered for inclusion on the basis of cost. 

The promotions officer then submitted the brief to three research agencies. 
She evaluated the agency proposals based on their understanding of the 
brief, the appropriateness and cost effectiveness of their proposed approach 
and their demonstrated expertise in the area. 
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Once the agency had been commissioned the promotions officer had to 
make, or at least approve, many decisions about the actual type of research. 
The agency could make recommendations but she had to take responsibility 
for those decisions. 
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Qualitative or quantitative research? 

The type of information required here was exploratory. The institute needed 
to understand how students made their choices. Accordingly, qualitative 
methods were judged to be the most appropriate. 

There was no point in trying to survey students on why, when and how 
they made their choices of institute because the researchers did not yet 
understand the dynamics of the decision process. A survey approach at this 
stage would generate data but it would be descriptive rather than analytic 
and would not answer the questions that the promotions officer was asking. 
It would actually raise more questions. 

Who are the respondents? 

Who should be included in the research? One obvious group was current 
students who had recently made the decision to attend this institute. The 
promotions officer thought it would be useful to also obtain insight into why 
some students did not choose the institute. First-year students from 
competing institutes should be included. Other questions considered by the 
promotions officer at this stage were: 

• Should final-year high school students be included? 

• Who else has an influence on the decision process - parents? teachers? 
peers? anyone else? 

, S3 



• Should respondents be restricted to the local area or be drawn from 
wider afield? 

Based on which groups could provide the most relevant information, the 
promotions officer and the research agency decided to include: 

• current students of the institute 

• current students of other institutes 

• parents 

• teachers. 

How should they be interviewed? 

While the promotions officer and the researcher had decided to use 
qualitative methods, they still had to decide whether to use group 
discussions or depth interviews. The choice of technique depended on how 
the promotions officer and the research agency believed decisions were 
made. They decided that the primary decision-making unit was likely to be 
the student and his or her parents, so decided to do depth interviews at home 
with the student and parents (the decision-making unit). 

Given that students may also be influenced to a fair degree by peer group 
pressure, the promotions officer and the research agency decided to 
supplement the depth interviews with some focus group discussions using 
affinity groups of students, so they could gauge the effect of group dynamics. 

For interviewing the teachers, a group discussion format was also seen 
to be the best choice because teachers were influences, not decision makers 
in this process. Their views and beliefs about how students make their 
decisions could thus be well aired in a group format. It was also efficient 
in terms of time and money. 

GVD a on a gj 0 on gj tithe [process 

Having made the above decisions, the promotions officer waited while the 
research was conducted. During this period she was in regular contact with 
the researchers and was kept informed about progress of the research. She 
also asked for regular progress reports which she passed on to her manager. 

Reporting and presentation 

The promotions officer had arranged to have the report presented by the 
agency to the senior management team of the institute. It was important that 
they should know the results of the research and be able to ask questions 
directly of the research agency. 
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Using the results 

With a bank of information on why, when and how students choose an 
institute, the promotions officer had a sound basis for gauging what 
communication strategies were going to appeal to students, which channels 
of information were most important to them, how much influence parents 
have on the process, how much influence teachers have on the process, and 
at what time of the academic year most of these decisions are being made. 

The promotions officer was well equipped with solid information on 
which to base her promotion and publicity decisions. She could be confident 
about selecting the most effective messages in the most effective media, 
directed to the most relevant decision makers at the right time. 
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SUMMARY OF CASE STUDY 1 



Problem definition How can we make best use of promotion and 
publicity? What messages should we send? 
To whom? Using which media? At which time 
of year? 



Information 

requirements 



How do students decide which institute to attend? 
What 'features’ do they use to compare courses 
and institutes? Who else is involved in helping make 
the decision? 



Is there a need 
for formal 
market research? 



Yes. We do not understand sufficiently well how 
students decide which institute to attend. 

The pattern may have changed in recent years. 

Our spending on advertising and promotion 
warrants investing in research to direct it effectively. 



Internal or external 
resources? 



Commission research externally. Internal staff lack 
expertise in research techniques or do not have the 
time to dedicate to the project. 



The research brief 



Introduction describing the background to the 
project and providing institute context. Statement of 
management decisions to be made as a result of the 
project. Statement of research objectives. Indication 
of budget available and expected completion dates. 
Request that research agency provide detailed 
description of methodology and rationale for usage, 
and costing of project by task. Brief submitted to 
three agencies. 



What type 
of research? 



Managing the 
process 



Qualitative data will be collected from current 
first-year students at this and other institutes, 
final-year high school students and their teachers. 
Focus groups will be used for teachers and high 
school students, and depth interviews with current 
first-year students. 



Selection of research agency, agreement on 
agency’s approach and costing, and regular 
progress meetings whilst research is conducted. 
Arrangement of formal presentation of results by 
research agency to senior managers. 
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The director of the institute was concerned about enrolments. Whilst overall 
enrolments seemed reasonably healthy, in the previous couple of years a 
slow but steady decline had occurred in enrolments in many recurrently 
funded courses. The director was becoming concerned with this decline in 
the programs he regarded as the ‘bread and butter’ of the institute. 

DtniffoxtnrDatiiiotn) tr<s a tr© mortis 

• Had student needs or the profile of students changed in the local area? 
(Was the market moving to non-recurrent programs?) 

• Were students going elsewhere because other providers had a higher 
profile or were seen to have better courses? 

• Were institute staff focusing their energies on non-recurrent programs at 
the expense of recurrent programs? 

Ds tilh)©ir© a on©©(rf] ffotr ff©innraa)0 max Ik ©ft (r©s©ai(r©[h)T 

The director wasn’t sure why enrolments were declining but he knew he 

couldn’t orchestrate remedial action without more information. 

Informal or formal research? 

The director decided that first of all he should harness internal resources 
and expertise. He asked staff who managed recurrent and non-recurrent 
programs, as well as those involved in advertising and promotion, to prepare 
for a workshop to ‘brainstorm’ potential reasons for the change in 
enrolments. 

As part of the preparation for this session, he asked a staff member to look 
at several existing data sources to see if these shed any light on the problem. 
The following data was analysed: 

• Data from the Australian Bureau of Statistics and from the Commonwealth 
Department of Employment, Education, Training and Youth Affairs, to 
look at any changes in demographics and training demand. (Was the 
market changing?) 

• The institute’s own data bank on past students, to find out whether the 
demographic profile of students had changed. (Did the institute’s client 
base match the overall market or was it skewed in certain ways?) 

• The institute’s own data bank on student satisfaction with recurrent 
courses, to find out whether the quality of recurrent programs had 
diminished. (Were enrolments falling because quality had declined?) 





The workshop was very productive. The data analysis and staff feedback 
helped the director narrow down the potential reasons for the drop in 
enrolments. The internal course satisfaction data suggested program quality 
was not an issue (satisfaction had not been declining). However, the 
comparison of external data with the institute’s student data base suggested 
that even though there was a strong market amongst school leavers, the 
institute’s enrolments were declining in this market segment. Staff managing 
programs agreed they had been putting more energy into chasing new target 
markets in the past couple of years at the expense of the school-leaver 
segment. 

However, staff at the workshop agreed they still had gaps in their 
knowledge with regard to the school-leaver segment. In particular, they did 
not know: 

• the types of courses being considered by the current batch of final-year 
students; 

• the types of institutes being considered by final-year students; 

• how the institute was regarded by school leavers and how the institute 
compared to competitors. 

Thus the workshop enabled the director to identify gaps in the institute’s 
collective knowledge, and at whom any further research should be aimed. 

Don -[fa ©das© [rosoatrclfa ©tr ©dottsomc"©©^ trosoaitrcfalP 
The director knew the institute had conducted its own research before, and 
had staff who were quite expert in developing questionnaires and doing 
telephone and mail surveys. Indeed, the institute captured quite a lot of data 
about its current students via telephone and mail surveys. The institute was 
also fortunate to have a staff member who knew how to collate and analyse 
the data using the computer program SPSS. 

The director organised a meeting with relevant staff to discuss an 
appropriate research methodology and whether it was more cost effective 
to conduct the research in-house or externally. 

The staff at the meeting decided to commission the research. 

• They didn’t want to bias responses, and hence thought the research would 
be better conducted by an independent third party, with the name of the 
commissioning provider not being divulged to respondents. 

• Whilst they had the resources to conduct research with current students 
(who were obviously easy to identify and contact), they did not have the 
required resources to find and interview a reasonably representative 



sample of final-year students within their catchment area. This would 
require a lot of interviewers who would need to work in the early evening 
and at weekends (in order to ‘catch’ students). 

Finding and briefing an agency 

This was easy. The institute had commissioned research in the past and had 
formed a good working relationship with a particular supplier. Quotes would 
be obtained from two other agencies (based on exactly the same 
methodology). 

©1? ir<es<ea)[r©lh)1P 

Given the institute had quite simple information requirements, the institute 
and research agency agreed that telephone interviews would be appropriate, 
cost effective and would suit the timetable (for which information was 
required quickly). Based on the research objectives, and the institute’s 
knowledge of the market (for example knowledge of potential competitors 
and potential courses), a structured questionnaire was drafted and piloted 
with 30 respondents. 

Apart from ensuring the questionnaire was effective and complete, the 
pilot also enabled the research agency to discover the incidence of final-year 
students within households in the agreed geographic catchment area. This 
meant field costs could be finalised. (Previously, they had been based on a 
best estimate of the incidence). 

Once the pilot was complete, and final revisions had been made to the 
questionnaire, a total sample of 800 final-year students was achieved within 
the catchment area. This sample size meant the director could have a high 
level of confidence in the aggregated results and, more importantly, could 
break the sample down into smaller groups (for example to compare students 
from public and private schools or to exclude students who wanted to study 
courses not offered by the institute). 

(Ml a mj a 0 mj © ttthi© pmcess 

Reporting and presentation 

In order to contain costs, and because a staff member knew how to specify 
and analyse cross tabulations, the director did not ask the research agency 
to provide a written report. Rather, the institute asked for the data to be 
provided in the form of cross tabulations which were then analysed 
internally. 
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Using the results 

Once the data was analysed, the director had an informed view on: 

• the types of courses in demand amongst final year students; 

• the key features sought in a course; 

• the types of providers being considered by students (which indicated who 
the main competitors were; 

• the key features sought in a provider; 

• who students took advice from, and which source of advice was most 
valued (the director had been particularly keen to assess the importance 
of the student career counsellors in secondary schools in order to decide 
whether the institute should target them more energetically); 

• the institute’s positioning relative to key competitors (achieved by asking 
respondents to rate different providers on a series of important features 
and to rank providers based on their overall preference). 

The results from the study were very enlightening, and provided the director 

with information for developing strategies to reverse the downward trend 

in the institute’s enrolments. 



SUMMARY OF CASE STUDY 2 



Problem definition 


Declining enrolments in recurrent programs. 


Information 

requirements 


Need to determine the causes of decline. 


Is there a need 
for formal 
market research? 


Need exists for formal and informal research. 
Preliminary informal research consists of 
brainstorming workshop with key staff and analysis 
of existing data to narrow down the range of 
possible causes of decline. Informal research 
identifies school leavers as an important market 
segment which has been neglected and which 
needs to be better understood. 


Internal or external 
resources? 


Decide to outsource in order to protect against 
respondent bias and to overcome resourcing 
problem of identifying and interviewing a large 
number of final-year students. 


The research brief 


Standard brief to a single research agency. Once the 
research methodology is finalised, further quotes 
test the competitiveness of the initial quote. 


What type 
of research? 


Telephone survey, using structured questionnaire 
developed using pre-existing knowledge within the 
institute. Questionnaire piloted to ensure 
effectiveness and to establish incidence of final-year 
students within catchment area. Total sample of 800 
provides reliable data, and allows for results to be 
broken down into sub-groups. 


Managing the 
process 


Decision to split the process to contain costs, and 
to take advantage of internal expertise and facilities. 
The research agency conducts the pilot and the 
survey, and presents the results in the form of 
cross-tabulated data. Expert staff within the institute 
analyse the data. 
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The principal sensed that some short courses offered by her private institute 
were underpriced. Similar short courses offered by some of the universities 
were significantly more expensive, and her analysis of course content 
suggested these courses were not offering anything different. 

Dmffotrmaiioom [recqiuiotrememtis 

The principal felt confident that her courses were of high quality. Course 
assessments conducted with students and sponsoring employers were 
always very positive. She insisted that formal measures of student 
satisfaction were kept and monitoring had indicated that they were 
consistently high. So the principal knew she had ‘a good product’. 

However, she also knew she could not afford to raise the fees if that were 
to jeopardise the number of enrolments in short courses. She did not have 
the luxury of excessive demand to protect the institute against a fall in 
enrolments. She needed to know whether and by how much she could 
increase the fees without suffering a drop in enrolments. In addition, she 
needed to make sure she did not alienate some key clients who provided 
her with a constant supply of students. 

Os Hihietre a meed] tfotr formal matr^eli ire s@ sure Oh*? 

Informal or formal research? 

The principal had canvassed the opinion of staff, and every staff member 
seemed to have a different point of view! Whilst she felt confident her staff 
were very close to the market, which meant they were quick to recognise 
new opportunities and to meet client expectations, she also knew that 
determining a market’s price sensitivity was a tricky business. The institute 
just didn’t have the knowledge to help her make a decision. She needed 
specialist information. 




Dm -Duo das© treseatrcOn ©ir ©nail sons [reed treseatrcOnT 
This decision was an easy one to make in one sense. The principal simply 
did not have a budget available to commission an external agency. However, 
she did have systems in place which allowed the institute to capture a lot 
of data about people who made inquiries about courses and about the 
students themselves. 

Another reason for doing the research in-house was that none of her staff 
had experience in briefing or managing an external agency and she did not 
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know which agency to approach. The principal felt more comfortable using 
in-house resources, though she realised this would present some limitations. 
She knew she would have to carefully define the scope of the research, and 
not be too ambitious in her expectations. For example, while it would be 
good to talk to a wide range of potential students the principal realised she 
would have sufficient resources to talk only to those who actually inquired 
about her institute’s courses. 

The principal also realised that in-house research would take longer to 
conduct at both data-collection and analysis stages, and she would need to 
streamline collation and analysis. (The last thing she wanted was a huge pile 
of completed questionnaires on her desk and to be wondering what she 
should do next.) 
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The principal decided there would be two phases to the research. The two 
would run concurrently. The first phase would involve the principal and 
other senior program managers talking to employers who currently 
sponsored students. 

The principal thought of these interviews as qualitative depth interviews, 
but they would provide a second benefit of allowing her and other staff to 
catch up with some important contacts. In conjunction with her staff, the 
principal prepared a discussion guide which covered: 

• how the employer felt about different aspects of the courses, and how 
satisfied the employer was overall with the courses; 

• whether the employer was using similar courses from other providers and, 
if so, how the institute’s courses compared both in terms of quality and 
price; 

• how the employer felt about the current pricing of the courses, whether 
the courses were regarded as cheap, good value, the same as comparable 
courses, a little more expensive than other courses, or a lot more 
expensive; 

• how the employer felt about an increase in fee pricing, and how the 
employer felt about different price points (the principal knew this part 
of the interview would need to be handled with sensitivity, but hoped 
that some of the earlier discussion would provide guidance); 

• whether there was a need for volume discounts, which could be used to 
offset some of the impact of price increases with employers who provided 
a lot of business. 

The principal decided to pilot the approach herself by doing a couple of 
interviews with employers she had a good relationship with. 




The second phase 

Over the years, the institute had developed an extensive data base, as the 
principal had ensured that information about students and inquirers was 
collected systematically and analysed regularly. Developing systems to 
collate and analyse the data had been a case of trial and error over the years, 
but the principal was now confident the institute had good systems in place. 
Indeed, the analysis, a regular agenda item at the institute’s fortnightly 
management meetings, had often indicated emerging opportunities. 

The second phase of the research would involve adding some pricing 
questions to the data collection sheets used for all people who inquired 
about courses (either in person or via the telephone) and for enrolling 
students. The principal had done some reading on how to ask questions 
relating to pricing. There was no definitive answer, so she decided on the 
following approach: All inquirers and enrollers were to be asked: 

• at what price they thought the course represented good value for money; 

• how likely they would be to enrol at this price; 

• at what price they thought the course was starting to get a bit expensive; 

• how likely they would be to enrol at this price; 

• at what price they thought the course was too expensive to consider. 

In addition, inquirers and enrollers were also to be asked what other 
courses they had seriously considered (or were seriously considering). This 
would provide comparative price points, as it was easy to ascertain what 
competitors were charging. 

0^0 a tro si g) d tro gj tifhie process 

The principal piloted the qualitative depth interviews herself, provided 
feedback to senior staff, and then suggested staff use the approach with their 
own contacts amongst employers. Once everyone had done a couple of 
interviews, the principal called a meeting with staff to discuss the emerging 
picture and to talk through any potential problems staff had with the 
approach. Some further interviews were then conducted, followed by a 
workshop, where the results were discussed in more detail, along with their 
implications. 

The principal decided the questions to be asked of inquirers and enrollers 
would be trialed for one month and then reviewed. Meanwhile she kept in 
touch with staff administering the questions to get their anecdotal feedback 
on how the approach was working. 

After one month, the results were collated. As it had been quite an active 
period for short-course enrolments, the sample was reasonable, although 
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not large. The results provided some interesting insights, suggesting that for 
certain short courses, the institute was indeed competing with the 
universities and was regarded as low-cost by both employers and students. 

The principal decided to collect more data before making a decision to 
increase the pricing, but she was most encouraged by the information which 
was coming through. 



SUMMARY OF CASE STUDY 3 
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Problem definition Should we raise the fees for our short courses 
(without adversely affecting demand)? 

Information Need to determine who the courses are competing 

requirements with, and the price sensitivity of the market (students 

and employers). 

Is there a need Need exists for formal research, as there is little 

for formal data or knowledge on price sensitivity within the 

market research? institute. 



Internal or external There is no budget available to commission an 
resources? external research agency. The institute does have 

good systems in place to capture data from 
‘inquirers’ and ‘enrollers’ and senior staff have 
good contacts among employers. 



What type Qualitative depth interviews among employers 

of research? (conducted by senior staff). Addition of pricing and 

competitor questions to standard data sheet used 
for all inquirers and enrollers (either face-to-face or 
via the telephone). 



Managing the Pilot of the qualitative and quantitative (data sheet) 

process approaches with regular feedback loops. Analysis of 

qualitative interviews in a workshop session. 

Analysis of data sheets after one month, with the 
intention of continuing the approach and building up 
the sample. 
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Several senior staff in the institute had recently been to a short in-service 
course on marketing post-secondary education. Having found the course to 
be both informative and helpful, they were determined to put some of what 
they had learnt into practice in their institute. 

While they agreed that the institute had no major current problems in 
relation to enrolments, particularly in recurrent courses, they all felt that it 
would be a mistake to be complacent. The course had demonstrated to them 
the advantages of being proactive to avoid trouble in the future. 

One area in which they believed that they could improve their systems was 
in tracking student satisfaction. They did ‘measure’ some aspects of student 
satisfaction within the institute now but the ‘measurements’ were fairly ad 
hoc, very little was done with the results and there was no consistency 
across the system in terms of whether satisfaction was to be measured and, 
if it was, how it was to be measured. 

Most of the student satisfaction measures that were being taken were in 
the form of course evaluations. This group felt that they needed a wider 
range of satisfaction measures to understand student requirements. Some 
of the areas they believed should be covered were: 

• provision of courses 

• timetabling of classes 

• teaching methods: course content, course assessment, course organisation 

• library services: accessibility, range of resources, level of assistance 

• cafeteria services: hours of opening, range of foods, pricing, service 

• childcare services: availability, hours of opening 

• career counselling: activities of career counsellors, range of services 
provided, effectiveness of services provided. 

Ds ftlhotr© © rD©©d] ffcoxr ffotnrtroaiO matrfcoft ir©s©anr<D[h)1P 

This group of senior staff decided that they should standardise formal 

systems for measuring student satisfaction and set up systems for using the 

results. 

Informal or formal research? 

The senior staff who had attended the course had a preliminary meeting to 
discuss how they should now proceed. They had all agreed that the institute 
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needed a more systematic method of measuring student satisfaction across 
a number of areas but what was the best way to achieve this? 

The group was aware of a number of different areas in which student 
satisfaction was already being measured. Course evaluations were routinely 
carried out in their departments. However, it quickly became apparent that 
no one knew how many ‘evaluations’ were being carried out and that an 
overview of satisfaction levels across courses would require a common 
measuring instrument. It was decided to undertake formal research to put 
all of this ‘measuring activity’ onto a more systematic basis that could 
generate results that could be acted upon at an institute level. 

At this preliminary meeting it was also determined that one person should 
take charge of the project. The head of the Business program was chosen as 
project manager because he had a background in research and a keen interest 
in developing an effective method of tracking long-term student satisfaction. 
His brief was to manage the process, utilising resources inside or outside 
the institute and report back to the meeting once he had prepared an 
assessment of needs and recommendations on how to meet them. The senior 
staff who had undertaken the marketing in-service course were to form an 
advisory committee which would meet at regular intervals to review 
progress on the project. 

The meeting also considered how broadly student satisfaction should be 
measured. Staff felt that in the past they had tended to think about student 
responses only in terms of course evaluations. They decided to extend the 
measures of satisfaction to general services such as library, childcare, 
cafeteria and counselling, as these services mattered to students. 

Oini-thoMS© oir ©cLa^soiiaircocd] irososircihlP 

The project manager was aware of a number of satisfaction surveys already 
being conducted within the institute. Being experienced in conducting and 
managing research he knew how to tap the considerable amount of expertise 
in constructing data bases and analysing data within the institute. He 
decided to conduct the research in-house. 

Apart from the cost considerations, the project manager thought that it 
would be more effective to conduct the research in-house because staff and 
students would be better placed to list the attributes that contributed to overall 
student satisfaction than would a group of external consultants. He also knew 
that institute staff, in particular, would have the best knowledge of the range 
of services that could be offered to students. He felt that external consultants 
would need considerable time to understand the institute sufficiently well to 
be able to do as good a job as would internal staff, and considered that the 
external perspective would offer, little added value to the project. 





With experience in constructing survey instruments, the project manager 
knew that the best way to achieve a satisfactory instrument was to take a 
two-phase approach to the task. The first phase of the research would be 
qualitative. It would be directed to developing an understanding of all the 
issues involved in student satisfaction. The second phase of the research 
would be quantitative, and comprise the construction of the survey 
instrument, piloting of the instrument, use of the instrument, analysis of data 
and reporting of results. 
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Qualitative research 

The workshop 

The project manager decided to start the process by conducting a workshop 
with institute staff who were currently administering their own course 
evaluation surveys. A workshop would be a good way for them to share 
information and for him to be able to get to know what student satisfaction 
measures were currently collected by the institute, where data was held and 
how it was used. 

The workshop also provided an opportunity to introduce a range of staff to 
the idea of a more systematised and institute-wide form of student 
satisfaction survey. The workshop was an occasion on which the new system 
could be ‘sold’ to staff and staff input could be used in the development of 
the system. The workshop went to plan. 

Focus groups 

A number of focus groups were conducted with institute staff and with 
institute students to develop a comprehensive ‘listing’ of the relevant 
attributes that contribute to ‘satisfaction’. The project manager employed a 
professional external moderator to conduct these sessions because he 
believed that he was less likely to get biased responses that way. 

Quantitative research 

The focus groups produced a large number of attributes of student 
satisfaction. They also indicated that two different types of questionnaires 
were required. The first type of questionnaire should be course-specific, to 
provide satisfaction measures on specific aspects of specific courses. It could 
be used at the completion of every course. The second type of questionnaire 
was a more general one that could be used annually to gauge student 
satisfaction with the institute on a more general basis. The project manager 
translated the general satisfaction attributes into specific questions and 
developed two pilot questionnaires. 
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The pilot 

Both questionnaires were piloted with a small number of students who were 
asked not only to complete the questionnaires but also to comment on how 
relevant, credible and ‘user-friendly’ they found them. Students were also 
asked to provide suggestions for changes to the pilot questionnaires. 

As a result of the pilot, a number of changes were made to the 
questionnaires. A second and larger pilot then followed. The data collected 
from those questionnaires were analysed and a number of different types 
of reports generated. These reports were then circulated to all heads of 
department and the senior administrative staff for comment. 

Using the results 

The survey results were analysed to develop benchmarks for current levels 
of student satisfaction. The analysis also highlighted areas of high 
satisfaction and areas of dissatisfaction. This information was then 
disseminated to the relevant staff for comment and recommendations for 
action. 

Use of the questionnaires became a regular institute activity for measuring 
shifts in student satisfaction both at a general level and on specific issues. 
As a result of developing an institute-wide student satisfaction survey 
system the institute was better placed to: 

• understand the causes of satisfaction and dissatisfaction amongst their 
students; 

• improve its performance in areas where students expressed dissatisfaction; 

• reinforce or develop its performance in areas where students expressed 
satisfaction; 

• build a longitudinal data base of measures of student satisfaction with all 
aspects of the institute, that would be useful in long-term planning; 

• better differentiate between issues that caused temporary if intense 
dissatisfaction and those that were more seriously undermining the 
standing of the institute; 

• provide benchmarks for all institute staff on appropriate levels of ‘service’ 
to be provided to students. 

The senior group who had started the project were very pleased with the 
outcomes of the research and felt confident that they now had a better grasp 
of what their students were seeking in the institute and how to meet those 
needs. 





SUMMARY OF CASE STUDY 4 



Problem definition 


How satisfied are our students? 


Information 

requirements 


Need to determine general levels of satisfaction of 
students with both educational issues in the institute 
(provision of courses, timetabling and teaching of 
those courses) and with other services supplied 
(library services, canteen sen/ices, career 
counselling and other student services). 


Is there a need 
for formal 
market research? 


Formal, objective research should supplement 
existing fragmentary and anecdotal evidence of 
levels of student satisfaction. 


Internal or external 
resources? 


There are few funds available for this research. 
Further, it is believed that internal staff will have a 
better idea of the relevant attributes of service that 
should be measured. 


What type 
of research? 


Workshops amongst institute staff and institute 
students to develop a listing of relevant attributes 
that contribute to ‘satisfaction*. Development of 
survey questionnaire(s) for academic and non- 
academic sen/ice issues. Continuing, regular 
measurement of levels of student satisfaction to 
build and renew data. 


Managing the 
process 


Workshop to consider the format of the survey 
questionnaire, followed by a pilot. Analysis of survey 
results and development of benchmarks for current 
levels of student satisfaction. Repeat of 
questionnaire on a regular basis to measure shifts in 
student satisfaction, overall and in specific areas 
over time. 
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The newly appointed director of this recently amalgamated multi-purpose, 
multi-campus institute had been director of one of the component institutes 
of the merger. He felt confident that he had a good grasp of the issues that 
were important to the new institute and of the concerns of members of the 
old institutes regarding the amalgamation. 

The first task facing the new director and his senior managers was to agree 
on a new name for the institute. This was proving to be a more difficult task 
than he had envisaged. There were major internal issues, for example staff 
members from the former institutes were keen to see some continuity of the 
identity of their former institute in the new name. There were also significant 
external issues. The director believed that few people in the local 
community realised what the amalgamated institute had to offer and how 
it would work. All the changes in the system recently had probably left the 
local community confused about the institute. 

The second issue of concern to the director was how to manage public 
relations and promotions activities for the new institute. Because of all the 
changes, he no longer knew how the community viewed the institute. He 
suspected that community images of the institute would be very confused. 

The renaming of the institute would be an important symbolic beginning 
for the new institution. It was also going to be an expensive exercise. 
Consequently, it was important to make sure it was done well. 

The director sensed that priority should be given to better understanding 
the image of the institute in the community. The renaming exercise could 
then be used to best advantage to help create a positive and favourable image 
of the new institute in the community. 

Om If ©Erma ft com 

Some of the things that the director felt that he, and his senior staff, needed 
to know were the following: 

• Current images: Just what were the current images of the institute within 
the community? 

• Development of image: How did the current images develop? What were 
they based on? How accurately did the current images reflect the reality 
of the institute and what it had to offer? 

• Impact of image: How much did the image of the institute influence 
peoples behaviour in relation to institute use? 




Differing images: Did different sub-segments within the community have 
different images of the institute? If so, what were these? What impact did 
they have? 
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Informal or formal research? 

Traditionally, institute directors have believed that they had a good grasp 
of the relationship between the institute and the community. They 
developed this understanding through their close personal ties with the 
community and by being represented on community boards and committees. 
The ties were both formal and informal. This type of arrangement had served 
the institutes well. 

However, with amalgamations and the increase in the number of 
alternative vocational education and training institutes, the picture was 
changing. The director of a newly amalgamated institute was inheriting a 
combination of images of the former institutes which were not always 
compatible with the new image being projected. 

A further complication was that the institutes themselves had changed 
markedly in the range and style of courses and programs that they were 
offering. In many cases, the community had barely kept up with the pace 
of change. 

Consequently, the traditional intuitive grasp of the position and image of 
the institute in the community was no longer sufficient to account for all 
of the changes. A more formal approach was needed. 

0 cm -(hi® qjs© treseardhi or ©tuttsotunrcod] trees ©a troth*? 

Although there were a number of staff within the institute who were well 
versed in undertaking research there were reasons, in this case, for favouring 
using external consultants. 

• Politics: Because of the newly amalgamated nature of the institute it was 
felt that using external consultants would help to ensure that the results 
would be acceptable to all interest groups within the institute. 

• Avoidance of bias: External consultants were more likely to get direct 
responses from participants in the research about the image of the institute 
than might be expected if institute staff were gathering the data. 

• New perspective: External consultants could be expected to provide a new 
perspective on the institute. They have no personal history of the institute 
to influence their understanding of the issues. 

Tlh® [roseatrclh fotriioff 

The director had little experience of using external agencies for research. 
In his former institute he had not been faced with the complex demands that 
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he was now facing. Decision making had been based on an intuitive 
understanding of the issues supported by some in-house research. In this 
new situation he was unsure of how to engage an external company. 

He first telephoned other directors of institutes to ask them for 
recommendations. As a result of his telephone calls he came up with a short 
list of companies to approach and, just as importantly, a checklist of what 
to look for in an external research company: 

• Professional standing: This is evidenced by membership of the 
professional bodies such as the Market Research Society of Australia. 

• Track record: Some indication of track record in terms of current and past 
clients. 

• Qualifications: Level of qualifications of staff employed in the agency. 

• Expertise: Indication of expertise in the areas that you are wishing to 
research. This does not mean that an agency would need to be expert in 
researching the vocational education and training market but rather that 
they have some understanding of the area and, more importantly, be 
expert in the research techniques that are most suited to researching your 
project. 

• Approachability: You need to feel comfortable with the contacts that you 
have in the agency. A successful outsourced research project is dependent 
on good working relationships between client and consultant. You should 
expect effective interpersonal relations with your consultant. 

• Mode of working: Check to see who will actually be doing your research. 
Some agencies assign a project manager who does the client liaison while 
someone else does the actual research. In other agencies the project 
manager handles all aspects of the research from client liaison, to data 
collection, to report writing and presentation. 

• Costing: Cost always remains an important consideration. However, it is 
not necessarily the case that the lowest priced proposal will produce the 
most effective results. The overall price of the project should be measured 
against the long-term cost of incomplete or inadequate results. The 
criterion here should be value for money rather than a simple measure 
of cost. 

Equipped with this advice the director set out on his search for an 
external agency. The director was advised to pre-brief agencies before 
sending out a formal brief. Pre-briefing enables the agency to write the best 
possible proposal. The better an agency understands your requirements, the 
better they can tailor a proposal to your needs. Additionally, a pre-briefing 
contact with the agency gives some insight into the type of organisation with 
which you are dealing. 




The director was advised that if you have done your preliminary work 
well you should only need to brief three agencies. Evaluation of proposals 
requires a lot of work on the part of the institute decision makers and to have 
too many proposals to consider can result in skimping the evaluation 
process. 

The director was inexperienced in commissioning market research so he 
ensured that his brief to the agencies was as clear and concise as possible. 
He was interested in seeing what they could come up with in terms of 
proposed methodologies for the project rather than dictating a methodology 
to them. Nevertheless, he was very clear about the background to the project, 
the overall research objectives, the decisions for which he needed the 
research results, his timing requirements and the general range of the budget 
for the project. 

WDnsott oU [rosea crctft) IP 

Once the agency had been commissioned the director had to make, or at least 
approve, many decisions about the actual type of research. The agency could 
make recommendations but he had to take responsibility for those decisions. 

Qualitative or quantitative research? 

The type of information required was exploratory. The institute needed to 
understand the nature of the images that the community had of it. 
Accordingly, qualitative methods were chosen. There was no point in trying 
to survey members of the community on the types of images they had of the 
institute because that would not help the institute to understand how these 
images had been formed or how to alter or recast them. 

A survey approach at this stage would generate data but it would be 
descriptive rather than analytic and would not answer the questions that 
the director was asking. In fact, it would tend to raise more questions. 

Who are the respondents? 

Who should be included in the research? One obvious group was current 
students and staff of the institute. However, the director wanted to know 
more about how the institute was perceived in the community more 
generally, including by sub-groups in the community who were important 
to the institute. 

The director and the research agency decided to base their research on 
five groups: 

• Local business: The institute’s desire to increase its fee-paying courses 
meant that local business was a major future target. 
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• Local school staff: School leavers still made up most of the institute 
enrolments - so the views of teachers, career guidance officers and other 
educationalists were important. 

• Local government: The role of the institute in the local community was 
seen to be important to the institute and they wanted to know how they 
were perceived by this group. 

• Parents: Parents of students who are in their final years of high school. 

• School leavers: Students in their final year(s) of high school. 

How should they be interviewed? 

While the director and the researchers had decided to use qualitative data, 
they still had to decide whether to use group discussions or depth 
interviews. The choice of technique depended on how the director and the 
research agency believed images were formed. 

They decided that it would be valuable to conduct depth interviews with 
the business and government respondents. These two groups were likely to 
have formed specific images of the institute in different ways from one 
another. 

For the school staff and students, group discussions were regarded as the 
most suitable technique. There was likely to be a natural affinity between 
these groups and the images of the institute may well have arisen from the 
same influences. 
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While the research was conducted the director was in regular contact with 
the researchers and was kept informed about progress of the research. He 
also asked for regular progress reports which he could share with senior staff 
at their regular strategy meetings. 
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Reporting and presentation 

The director had arranged to have the report presented by the agency to the 
senior management team of the institute. It was important that they should 
know the results of the research and be able to ask questions directly of the 
research agency. 



Using the results 

With a bank of information about the way his institute was perceived by 
significant groups in the community, the director had a much better 
understanding of how images of the institute were formed by different 
groups. He had information on which to plan communication strategies that 
would appeal to particular groups. The institute possessed solid information 
on which to base its image building, renaming and promotion campaigns. 
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Problem definition 


What is the image of our institute in the community? 


Information 

requirements 


Need to determine the range of perceptions and 
attitudes that major client groups have of the 
institute. These groups include students, industry, 
parents, teachers and other members of the 
community. 


Is there a need 
for formal 
market research? 


There is a need for formal research. The institute 
has previously relied on its intuitive understanding 
of how the community perceived it. However, a 
number of changes in institute clientele, and in the 
institute offerings, mean that this intuitive grasp is 
no longer sufficient. 


Internal or external 
resources? 


Politically it is necessary to commission an external 
agency to conduct this research. There is too much 
chance of skewed results if the research is 
conducted by institute staff. It is likely that 
respondents will tell institute staff what they want 
to hear. An external researcher is necessary in 
this case. 


The research brief 


Pre-briefing calls to a number of agencies, and 
briefs to three agencies. Decision made on the basis 
of the quality and value of the proposals. 


What type 
of research? 


Research about respondent perceptions and images 
of the institute is best suited to qualitative research. 
Depth interviews with selected samples of the major 
client and community segments. 


Managing the 
process 


The research agency would manage the project in 

conjunction with an internal project manager to 

provide additional information when needed and to 

ensure that the project is kept to schedule and o 

budget. Close working relationship with agency > 

throughout the whole process. Agency to deliver full “ 

presentation of results and report, detailing results 

and providing conclusions and recommendations. co 
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